Our standard setup for clients is the Semantic Nexus — a design where the data architecture is based on our best practices and years of client feedback. Company leadership, finance teams, and team leads will use the Internal Analytics to track performance. Finance and planning managers rely on the semantic model to run ad-hoc calculations, which are all based on a single source of truth. After a succesful deployment of the Nexus, the monthly Company KPIs can then be directly shared with shareholders and stakeholders through the metrics exchange. ## Product Components ### Data Model Explorer #### Sources The majority of data analytics setups have a multi-source architecture, this implies combining the data from different software applications into one single truth. To make sure that the source data is properly structured and sufficiently filled out, the Internal Analytics design always has a Source section that summarises all major source tables. The source pages contain a singular table representing either a dimensional table or an event table, with the contents of the source pages showing the data after the transformation. A core component are the aggregation tables where the events are aggregated by event property, which allows for a fast deepdive in how the events are distributed across its properties and whether there are any irregularities. ![[Internal Analytics Sources.png]] *A typical source page with counters, measures, timelines, records and aggregations* #### Metrics Each company's success is built around a handful of metrics that must be meticulously tracked. In the case of MAXQ Analytics, the core metrics for our engineering business are: `billable_utilisation`, `availability`, `gross_margin`, and `booked_hourly_rate`. Each metric designated as critical to a company's success is given its own dedicated page. While the exact layout can be tailored to the client’s needs, we generally include a few scorecards that show: - The **current metric value**, based on any active filters. - The **numerator** and **denominator** used to calculate that value. Alongside the metric, the client provides textual context on the metric, the owner, universal business standards and the preferred bandwidth within which the metric should move. ![[Internal Analytics Metric.png]] *A dedicated metric page in Looker Studio ### Data Entry Tests The data entry tests is tracked using the *Data Entry Tests* which executes client specific tests on the dataset to make sure all data entries and business workflows are handling themselves accordingly. Each Data Entry Test receives an *Owner*, a *Deeplink* for resolving issues at the source, *Context Description* and a *Test ID* for ease of communication. ![[Internal Analytics Data Quality.png]] *Data Entry Tests as part of the overall Data Quality* ### Data Freshness Each data architecture contains a flowchart of all systems used in the data architecture ![[Internal Analytics Data Pipeline.png]] *Data Pipeline flowchart visualising the data extraction, transformation and consumption* These settings are especially usefull when refactors and expansions are considered, because then all stakeholders can see how the data model elements are defined and how the data architecture is currently organised. ### Domain Reports Within each of our clients we distinguish three business domains. Commerce, Finance and Operations. Each of these domains come with their own semantics, owners and targets. It is common practice to structure business in cost-centers linked to these domains, which enables a more granular budget and accountability governance structure. #### Commerce Atlas In the commerce atlas we present all metrics and measures relevant for the go-to-market motions linked to each of our clients. We provide periodic performance dashboards, accompanied by core process and trend analysis. ![[Commerce Atlas.png]] *Monthly Commerce Revenue Performance Dashboard* For trend analysis we create specific custom built pages focus on client requirements. ![[Commerce Unit economics.png]] *Trend analysis and specific unit economics per vertical per product* #### Finance Atlas In the finance atlas we present all metrics and measures relevant to the financial operations of each of our clients. We provide periodic performance dashboards, accompanied by core process and trend analysis. #### Operations Atlas In the operations atlas we present all metrics and measures relevant to the business operations of each of our clients. We provide periodic performance dashboards, accompanied by core process and trend analysis. ## Design Depending on the Analytics tool used, the design will look different. [[Looker Studio]] and [[PowerBI ]] allow for a menu-structure to be placed on the left-hand side of the report. [[Preset]] on the other hand, works with a per-dashboard approach, which is especially usefull when the number of report pages becomes too large to effectively fit in one report. ![[Looker Studio Report Demo.png]] *The report menu on the left side in Looker Studio shows a combined setup of the different Semantic Nexus components. ## Frameworks ### Standardised Data Architecture The SDA is one of three [[Frameworks]] that are put into place when designing a data arechiture. It provides us with the best practices to build a data pipeline and data model that is robust, flexible and low cost. Years of experience and mistakes have slowly provided us with this list that we believe creates a solid foundation. ### Integrated Performance Framework All the components of an effective performance tracking design require their place and relationship towards each other. In the IPF we have a systematised way of presenting the data model and make sure it strikes the right balance between simplicity and depth. ### Modular Security Protocol Our design preference for client-side and open core systems, makes the security environment a bit more complex than in a closed source monolithic architecture. Therefore we have a set of checks to make sure the security requirements are met. ## Communication ### Workload backlog The communication regarding all the outstandig tasks, their size and priority are tracked in the task management tool [[Trello]], but can also be tracked in other systems like [Monday](https://monday.com/) or [Jira](https://www.atlassian.com/software/jira). The main goal in the workload management is to create a continuous-flow process with a backlog that get's prioritised monthly. We structure the tasks through boards with distinct phases: - Backlog - Discussion (>4 hrs) - Discussion (<4 hrs) - Can be done - Doing - Review - Done In the two discussion phases we distinguish between minor and major tasks determined by a 4 hour treshold. All development tasks require green lighting by the Data Owner on the side of the client. Any minor ad-hoc issues resolve tasks and minor maintentance tasks are executed without the need for any formal decision making. To keep the process clean and the work-in-progress (WIP) at a manageble level, the number of tasks that need to be done, should not grow to big. First complete what you have already started vs. starting something new. ### Shared Chat Channel For all clients we have a shared chat channel, usually [[Slack]], but this can also be Teams. In this channel we have all relevant people present. The major updates, issues and bug-fixes are shared here, as well as requests from individual people go into this channel for all to see. The centralised communication approach makes sure there is no confusion or maojr back channel conversations. All topics data-related should be discussed there. ### Documentation The documentions or 'Docs', are filled with all key components of the Data & Analytics setup. Which are: - Architecture - Data pipeline (flowchart) - Data pipeline assets (link to asset overview) - Extraction - Computation - Transformation - Semantics - Analytics - Calculations - Data Model - Roadmap In each of the above compontents we write down all the relevant systems, assumptions, logic, owners etc. This allows us to have one central place that describes how the data architecture is organised. ### Code repositories We have a strong preference for version-controlled dedicated git-repositories. When it comes to custom forks of the [[Exact Online - Airbyte]] instance, the transfromation tool [[dbt]] or the semantic models in [[Cube]], all of them are stored in the client's [[GitHub]] or Gitlab account. We prefer this approach over having managed repositories at the vendor or repositories owned by our engineering company. We believe that each client should have full sovereignty over their architecture and should be able easily switch between vendor, as described in our [[Design Philosophy]]. ## Demonstration We published a [Data Model Explorer](https://lookerstudio.google.com/u/0/reporting/86bd9839-3788-4b62-a1d9-4ce312bbff3e/page/p_etg6w1911c) report for the State of Iowa Alcoholic Beverages Division, which reports on all 18 million invoices sent by the division since 2012. The data is stored in [[BigQuery]], where we have built the underlying data model using [[dbt]] and the semantic model with [[Cube]]. ## Pricing We offer the semantic nexus and all its components free of charge to all our clients. We only charge our standard hourly rate for its implementation.