Skip to main content
Ken Pendergast
04 / Data Platform20 TB

A scalable data platform and governance model.

StudentUniverse · 2020 – 2022

Delivered a scalable data platform and governance model following a nine-month implementation.

01

Executive summary

Led the implementation of Databricks as a core platform, processing complex GDS data with PySpark and managing 250+ nested JSON arrays alongside global PostgreSQL data.

02

Context and constraints

Awaiting verified detail

The starting conditions, constraints, and pressures behind this engagement are not established by the current source brief.

[ADD VERIFIED CONTEXT AND CONSTRAINTS]

03

Role and scope

Organization
StudentUniverseWaltham, MA · Dec 2020 – Dec 2022
Role
Business Intelligence Manager
Scope
Databricks as a core platform, processing complex GDS data alongside global PostgreSQL data.

04

System architecture

An abstract view of the transformation shape: disorder on the left resolving into governed structure on the right. It carries no proprietary schema, screen, or client detail.

05

Key decisions and rationale

Awaiting verified detail

The architecture decisions taken here, and the reasoning behind them, are not established by the current source brief.

[ADD VERIFIED KEY DECISIONS AND RATIONALE]

06

Implementation

Awaiting verified detail

How the work was sequenced and delivered is not established by the current source brief.

[ADD VERIFIED IMPLEMENTATION SUMMARY]

07

Outcomes and evidence

Delivered a scalable data platform and governance model following a nine-month implementation.

20 TB
Processed in Databricks and PySpark
250+
Nested JSON arrays managed

08

Technology and stakeholders

Technology

  • Databricks
  • PySpark
  • AWS S3
  • PostgreSQL
  • Tableau
  • ETL/ELT
  • Data Governance

Stakeholders

Awaiting verified detail

The stakeholder ecosystem for this engagement is not established by the current source brief.

[ADD VERIFIED STAKEHOLDER ECOSYSTEM]

09

Operating principles

Awaiting verified detail

The transferable lessons from this engagement are not established by the current source brief.

[ADD VERIFIED OPERATING PRINCIPLES]

Continue

Related transformations

All transformations