The solution consists of multiple ETL processes for data aggregation from multiple sources and building a set of Microsoft SSAS cubes to be consumed by various clients like Tableau and Excel.
The challenge was providing reliable and performant data source for data multiple visualisation tools, analysis and validation of a big amount of data, aggregate data from multiple source systems (SAP, various RDBMS, SalesForce, Splunk, WebServices, CSV, Excel, Sharepoint). After that, we needed to fit the daily data imports and processing into a 24 hours schedule, update the data on a daily basis for multiple world spread time zones. We also has issues with granting access to sensitive data based on authorisation roles and providing access to different teams in various stages.
Configure security mechanism to mask sensitive data depending on user access rights. Development of a very flexible and performant ETL solution for data aggregation from different sources. Development of Microsoft SQL Server Analysis Services cubes.
Flexible notifications for special events. 24x7 On Call Support provided by a dedicated team. Multiple environments for development, testing, stage and production. Provide an ad-hoc environment for other teams. Develop incremental data imports executed in parallel and add indexes where appropriate.
- SQL Server
- SQL Server Analysis Services
- SQL Server Integration Services, Reporting Services
- NetApp SnapManager for Microsoft SQL Server
- Tableau, Power BI, Excel
- Splunk, Pentaho
- Java, Groovy
- Junit, Jetty, Tomcat
- Jenkins, Maven
- Continuous Integration