Case studies

UK government agencyPublic sector

A national-security data platform, built in six weeks

We built a platform to ingest, model and search petabytes of investigation data, cutting lead times from weeks to hours.

An analyst at a bank of monitors in a dimly lit operations room.
to build the platform
6 weeks
to go live
6 months
to get data ready for investigation, down from weeks
2–12 hrs

The client

A UK government agency working on national security needed a data platform for its investigators. For security reasons we don’t publish the client’s name or the architecture.

The challenge

Getting data onto an investigator’s machine and ready for analysis took several weeks. The agency needed a platform that could ingest, model and visualise petabytes of data, keep running costs down, stay easy to maintain and give officers a simple interface.

What we did

  • Built an event-driven data pipeline on AWS ECS, SQS and S3, keeping running costs low.
  • Used Amazon OpenSearch to store, query and visualise processed data.
  • Added Amazon SageMaker for machine learning modelling.
  • Wrapped the platform in a bespoke Python front end with Okta single sign-on, so officers could work with it safely.
  • Automated and standardised the process end to end.

The result

The platform was built in six weeks and live within six months. Investigation lead times fell from weeks to between two and twelve hours, depending on the size of the data, while data quality, audit and provenance all improved.

More case studies

All case studies

Not sure where to start?

Tell us what you're trying to fix. A senior engineer will reply within one working day.