Data Pipelines: Workflow and Dataflow for Todays Data Architectures Training in New Zealand

  • Learn via: Classroom / Virtual Classroom / Online
  • Duration: 1 Day
  • Price: Please contact for booking options
We can host this training at your preferred location. Contact us!

Data-driven is the modern mantra of business management, but enabling a data-driven organization is complex and challenging. Abundant data sources and multiple use cases result in many data pipelines—maybe as many as one for each use case. Capabilities to find the right data, manage data flow and workflow, and deliver the right data in the right forms for analysis are essential for all organizations that seek to become data-driven.

Multiple and complex data pipelines can quickly become chaotic under pressure from agile development, democratization, self-service, and organizational pockets of analytics. The resulting difficulty in governance and uncertainty of data usage are only the beginning of the troubles. Therefore, data pipeline management must ensure that data analysis results are traceable, reproducible, and of production strength, whether enterprise-level or self-service. Robust pipeline management works across a variety of platforms from relational to Hadoop, and recognizes today’s bidirectional data flows where any data store may function in both source and target roles.

Analytics architects, BI architects, data warehouse architects, data architects, and anyone in an architect role that intersects with data; data engineers who define, design, and develop data warehouses, data lakes, operational data stores, data sandboxes, master data hubs, or other enterprise data stores; data integration and preparation professionals who define, design, and develop the processes that move data through pathways from sources to consumers.

  • The challenges and complexities of modern data pipelines
  • Why data flow and workflow are critical parts of—and how they fit into—your analytics architecture
  • How to define and design data pipelines
  • The roles and functions of metadata in pipeline management
  • The important relationships between pipeline management and data governance
  • The state of tools and technologies to support pipeline management

Part One: Today’s Data Challenges

  • Variety and Complexity
    • Sources
    • Ingestion
    • Persistence
    • Management Topology
    • Utility
    • Use Cases
  • Time to Value
    • Storing Data
    • From Origin to Destination
    • Finding Data
    • Learning about Data
    • Data Cataloging
    • Data Preparation
    • Analysis and Communication

Part Two: Modern Data Solutions

  • Growth and Scalability
    • Data Scalability
    • Process Scalability
    • People Scalability
    • Analytic Scalability
  • Rethinking Data Architecture
    • Persistence and Topology
    • Data Flow
    • Services
    • Governance
    • What does this mean for your architecture?
  • Building for the Future
    • Future of Databases
    • Future of Data
    • Future of Analytics

Part Three: Data Pipeline Design

  • The Big Picture
    • Pipeline Components
  • Destination
    • Purpose and End Point
    • Timeliness
  • Origin
    • Data Supply and Begin Point
    • Data Type and Velocity
  • Data Flow
    • Data in Motion
    • Pipeline Boundaries
    • Blending Batch and Real Time
  • Data Storage
    • Data at Rest
    • Choosing Data Storage
  • Processing
    • Data Products and Data Value
    • Ingestion
    • Persistence
    • Transformation
    • Delivery
  • Workflow
    • Sequence of Activities
  • Monitoring
    • Pipeline Health
  • Technology
    • Pipeline Tools
    • Abundance of Tools
  • Design Summary
    • 7 Steps


Contact us for more detail about our trainings and for all other enquiries!

Upcoming Trainings

Join our public courses in our New Zealand facilities. Private class trainings will be organized at the location of your preference, according to your schedule.

Classroom / Virtual Classroom
22 November 2024
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
27 November 2024
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
22 November 2024
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
27 November 2024
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
13 January 2025
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
25 January 2025
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
27 January 2025
Auckland, Wellington, Christchurch
1 Day
Classroom / Virtual Classroom
13 January 2025
Auckland, Wellington, Christchurch
1 Day
Data Pipelines: Workflow and Dataflow for Todays Data Architectures Training Course in New Zealand

New Zealand is an island country in the southwestern Pacific Ocean and it consists of two main islands and 700 smaller islands. Two main islands are the North Island and the South Island. The capital city of New Zealand is Wellington and the most popular city of the island country is Auckland. English, Māori and New Zealand Sign Language are the official languages of New Zealand. As of January 2022, the population of the country is about 5,138,120. 70% of the population are of European descent, 16.5% are indigenous Māori, 15.1% Asian and 8.1% non-Māori Pacific Islanders.

Since most of the country lies close to the coast, mild temperatures are observed year-round. January and February are the warmest months while July is the coldest month of the year. Fiordland, the first national park of New Zealand Tongariro

Unlock your potential in IT through our extensive selection of courses, which include programming, software development, data science, business skills, and cybersecurity. Our adept instructors will provide you with hands-on training and practical perspectives, all conveniently hosted at your desired location within New Zealand.
By using this website you agree to let us use cookies. For further information about our use of cookies, check out our Cookie Policy.