ScienceHub EarthCODE Feedback and Suggestions

Site Feedback
1

Thanks to everyone who joined the recent ESA ScienceHub / EarthCODE :earthcode: meeting! We discussed challenges like preprocessing at scale, aligning datasets, and improving data sharing. There’s also strong interest in data cubes, cloud computing, publishing data (ESA Open Science Catalog | ESA Open Science Catalog) and training on EO platforms.

Key Takeaway Notes From Our Interview

  1. Local Workloads: Most participants are still running their workflows locally, even for large datasets and extensive processing tasks.
  2. Diverse Tooling: There is a wide range of tools in use, such as QGIS, SNAP, GEE, Matlab, and Jupyter/Julia/Python.
  3. Heavy Workloads: Researchers are dealing with week-long processing jobs and handling diverse data formats, including CSV, NetCDF, DataFrames, TIFF, and Zarr.
  4. Low Awareness of Emerging Standards: Terms like STAC and data cubes aren’t well known.
  5. Training Needs: Researchers expressed a strong interest in learning to use platforms like DeepESDL, EDC, OpenEO and other cloud-native tools. They’re open to transitioning from local to cloud-based workflows but need more guidance.
  6. DOIs and Data Sharing: There’s a clear demand for better data publication workflows, including support for DOIs to facilitate citation and reproducibility.
  7. Computational Resources: Lack of access to sufficient computational power, especially for large-scale analyses, is a recurring bottleneck.
  8. Collaboration Features: Participants are keen on having collaborative spaces to share code and data within projects or research teams.
  9. Uncomfortable Workflows: Current workflows often require downloading processed data locally before re-uploading it to cloud platforms, which is inefficient and cumbersome.
  10. Complex Datasets Challenges: Processing complex datasets, such as Sentinel-1 SLC data, remains a significant challenge.

Let’s Keep the Conversation Going

What should we prioritize next and what are your suggestions about how EarthCODE can support you and your research?

We want to hear from you:

  • What should EarthCODE prioritize to address these challenges?
  • Are there specific features, training sessions, or resources you’d find most valuable?
  • How can we better support your transition to cloud-native workflows on EarthCODE?
2 Likes