DaaS Reference
Introduction
The BigChange Data as a Service (DaaS) leverages Snowflake's cutting-edge technology to securely put analytics-focused data in our customers' hands. The aim of this service is to enable our customers to generate impactful data-driven insights, thereby enhancing their capacity to make informed decisions based on data.
Our documentation is a priority to us and we strive to constantly improve it. Please reach out if you find an inaccuracy or have a suggestion on how we might make things better.
Setup procedure
Step 1 - Review our documentation
DaaS is a premium service that is not included in the standard JobWatch licensing agreement. Be sure to review the pricing information, data dictionary and FAQs to determine if the service is a good fit for your needs.
Step 2 - Create a Snowflake account
Snowflake is a market-leading cloud data warehouse provider that BigChange uses to provide DaaS. After you have signed up for a BigChange DaaS license, you will need to create a Snowflake account for your organisation, if you don't already have one.
After entering your name and company details, click continue and provide the following:
- Choose Standard for your Snowflake Edition
- Choose Amazon Web Services for your cloud provider
- Choose EU (Ireland) for your region
If your organisation requires use of a different region or cloud provider, we are able to support Snowflake accounts outside of AWS / EU (Ireland) for an additional cost. Let us know your requriement when signing up to DaaS.
Step 3 - Sign up to our service
You can sign up to BigChange DaaS by contacting your customer success manager or alternatively, by sending an enquiry from right here in the Developer Portal. Be sure to include the required information, including the identifier of your Snowflake account.
Step 4 - Accept the data share
We use Snowflake's data marketplace to securely share data with your own Snowflake account. We'll publish a private data listing that you need to accept and import to be able to access the data we provide.
- Sign into your Snowflake account, locate the BigChange privately shared listing and click the 'Get' button.
- We recommend leaving the database name and role as the form suggests. You can always change these later if you aren't sure. Click 'Get' to proceed.
- The share will be established and data will be made available immediately. Click the 'Query Data' button to examine it.
- The data marts will be presented in the database that you just created as a set of secure views. Expand the panel on the left hand side to locate and start interacting with them.
Available data
Schema
The BigChange DaaS license grants access to our collection of purpose-built data marts, geared for analytics purposes and aligned to key business entities and functions.
Our versioned marts adopt a wide table structure to ensure high performance querying and developer ease-of-use. Data heavy lifting has been eliminated by building complex calculations directly into the marts and minimising the the need for table joins.
Each data mart is fully documented, containing an easy to understand naming convention and human readable data values. Data professionals do not need to be familiar with any other BigChange database or product when working with DaaS.
Freshness
DaaS is primarily geared to provide retrospective analytical insight, rather than to drive real-time data use cases. At present, the data marts we surface are refreshed every 3 hours from Monday to Saturday during UK working hours (8:00-17:00).
Read only
DaaS provides data that is read-only and does not provide the ability to write data back into our platform.
We are continually building out the quantity and quality of the marts we provide, and we encourage you to regularly review the change log for updates. If your use case needs additional data, feel free to make a request via the developer community.
Using your data
BI Applications
The Snowflake technology that our DaaS is built upon is widely supported by most business intelligence applications. Below you'll find documentation from popular suppliers on how to connect Snowflake to their product:
| Application | Supplier Homepage | Documentation |
|---|---|---|
| Power BI | https://powerbi.microsoft.com/ | Snowflake Connectivity |
| Tableau | https://www.tableau.com/ | Snowflake Connectivity |
| Looker | https://lookerstudio.google.com/ | Snowflake Connectivity |
| QuickSight | https://aws.amazon.com/pm/quicksight/ | Snowflake Connectivity |
| Metabase | https://www.metabase.com/ | Snowflake Connectivity |
| Qlik Sense | https://www.qlik.com/us/products/qlik-sense | Snowflake Connectivity |
| ThoughtSpot | https://www.thoughtspot.com | Snowflake Connectivity |
Please note that while we encourage the use of BI applications, we are unable to provide support for these third party tools.
Programmatic access
If your requirements cannot be fulfilled by off-the-shelf business intelligence tooling, it is possible to build your own applications to interact with the BigChange DaaS. We have compiled the list below to provide a steer on how to get started with Snowflake programmatic interfaces for most common languages:
| Language | Interface |
|---|---|
| C# | Snowflake Connector for .NET |
| PHP | PHP PDO driver for Snowflake |
| Python | Snowflake Connector for Python |
| Go | Go Snowflake Driver |
| Node.js | Node.js Snowflake Driver |
| Java | JDBC driver for Snowflake |
| ODBC | ODBC driver for Snowflake |
Before building an application, please review our FAQs to understand the suitability of DaaS for your use case. Note that we are unable to provide support for any bespoke applications you build.
Data replication
Pulling data out
If your requirements mean that you have to merge data from DaaS with your existing data sources, you should consider establishing a synchronisation process.
Although you could create a custom application to transfer data from Snowflake into your intended database, we suggest leveraging an off-the-shelf SaaS solution for this task. FiveTran offers a cost-effective solution for this purpose, but there are several other platforms that offer similar services.
Pushing data in
Your Snowflake account is not BigChange specific and can be used to fulfill other data warehousing requirements for your organisation. In this regard, it might be more beneficial to explore the option of replicating additional data into your Snowflake account.
The same data transfer techniques outlined for moving data can also be employed with Snowflake as the destination rather than the source. Replicating data into Snowflake could be a preferable approach for blended insights, especially if your organisation lacks an established data warehousing capability.
Whilst your Snowflake account can serve as the source or target for data replication, such replication processes fall outside of the core service we offer. Please note that we are unable to provide support for the operation of third-party or bespoke applications for data replication.
Account management
You have complete control of how your organisation interacts with DaaS - BigChange applies no user or usage limits. You can manage your own Snowflake account to administrate user access, as well as scaling your warehouse to support your cost and performance targets. For more information how you can manage your account, please consult the Snowflake technical documentation.
Versioning
Latest data marts
Each of our data marts are versioned to provide a stable data contract for systems to be built upon. Details of our current versions are described below:
| Name | Latest version | Published on |
|---|---|---|
| Contact custom fields | contact_custom_fields_v1 | 2023-10-03 |
| Contact notes custom fields | contact_notes_custom_fields_v1 | 2023-10-03 |
| Contact notes | contact_notes_v1 | 2023-10-03 |
| Contacts | contacts_v1 | 2023-10-03 |
| Contract custom fields | contract_custom_fields_v1 | 2024-03-11 |
| Contract financial lines | contract_financial_lines_v1 | 2024-07-02 |
| Contract notes | contract_notes_v1 | 2024-03-21 |
| Contracts | contracts_v1 | 2024-03-04 |
| Invoice custom fields | invoice_custom_fields_v1 | 2023-10-03 |
| Invoice lines | invoice_lines_v1 | 2023-10-03 |
| Invoices | invoices_v1 | 2023-10-03 |
| Job constraints | job_constraints_v1 | 2024-11-13 |
| Job custom fields | job_custom_fields_v1 | 2023-10-03 |
| Job groups | job_groups_v1 | 2023-10-03 |
| Job lines | job_lines_v1 | 2023-10-03 |
| Job notes | job_notes_v1 | 2024-04-18 |
| Job worksheet answers | job_worksheet_answers_v1 | 2024-04-09 |
| Jobs | jobs_v1 | 2023-10-03 |
| Person custom fields | person_custom_fields_v1 | 2024-07-24 |
| Persons | persons_v1 | 2024-07-11 |
| Purchase order lines | purchase_order_lines_v1 | 2024-09-18 |
| Purchase orders | purchase_orders_v1 | 2024-09-18 |
| Quote custom fields | quote_custom_fields_v1 | 2023-10-03 |
| Quote lines | quote_lines_v1 | 2023-10-03 |
| Quotes | quotes_v1 | 2023-10-03 |
| Resource absences | resource_absences_v1 | 2024-07-11 |
| Resource activities | resource_activities_v1 | 2023-10-03 |
| Resource custom fields | resource_custom_fields_v1 | 2023-10-03 |
| Resource day summaries | resource_day_summaries_v1 | 2023-10-03 |
| Resource journeys | resource_journeys_v1 | 2024-07-02 |
| Resource notes | resource_notes_v1 | 2024-03-12 |
| Resource skills | resource_skills_v1 | 2024-11-18 |
| Resource working hours | resource_working_hours_v1 | 2024-08-12 |
| Resources | resources_v1 | 2023-10-03 |
| Sales opportunities | sales_opportunities_v1 | 2023-10-03 |
| Sales opportunity custom fields | sales_opportunity_custom_fields_v1 | 2024-01-09 |
| Sales opportunity notes | sales_opportunity_notes_v1 | 2024-10-30 |
| Stock item custom fields | stock_item_custom_fields_v1 | 2024-03-21 |
| Stock item notes | stock_item_notes_v1 | 2024-04-18 |
| Stock items to contracts | stock_items_to_contracts_v1 | 2024-10-17 |
| Stock items | stock_items_v1 | 2024-03-11 |
| Stock movements | stock_movements_v1 | 2024-04-08 |
| Stock planned usage | stock_planned_usage_v1 | 2024-04-08 |
| Stock reorder levels | stock_reorder_levels_v1 | 2024-04-23 |
| Stock suppliers | stock_suppliers_v1 | 2024-03-21 |
| Stock worksheet answers | stock_worksheet_answers_v1 | 2024-05-06 |
| Stock | stock_v1 | 2024-04-23 |
| Timesheet worksheet answers | timesheet_worksheet_answers_v1 | 2024-06-17 |
| Vehicle attributes | vehicle_attributes_v1 | 2024-03-27 |
| Vehicle checks and defects | vehicle_checks_and_defects_v1 | 2024-04-23 |
| Vehicle custom fields | vehicle_custom_fields_v1 | 2024-02-26 |
| Vehicle notes | vehicle_notes_v1 | 2024-03-12 |
| Vehicles | vehicles_v1 | 2024-02-26 |
Breaking changes
DaaS will evolve over time and whilst every effort is made to develop in an unimpactful way, there may be situations where we need to make breaking changes. We consider breaking changes to be:
- Renaming or removing an existing column
- Changing the data type of an existing column
- Changing the name or purpose of a mart
Where a breaking change is necessary a new version of data mart will be published and made available to you automatically. Older data mart versions will remain functional for a determined amount of time to ensure that existing systems can continue to operate uninterrupted.