Instructor: Adam Brazier

Description: An overview of managing data workflows for scientific computing, starting with data collection and aggregation, through processing and storing in an accessible form. We will cover some issues relating to security policy, integration of Identity and Access Management and retention policy (but this is not a security policy workshop!), possible storage venues and formats, models for aggregating and distributing data such as the pub/sub model, and modes of data storage such as relational database, file system, cloud, noSQL, Data Lake, etc.

Prereqs: Some knowledge of software and data processing