DaaS FAQ
Please reach out to us via the BigChange Developer Community and we'll be happy to answer your question.
Usage Guidance
Who is DaaS for?
DaaS is for data professionals to provide analytical insight on the performance of their organisation. Generally, this will be for production of reporting/dashboards using off the shelf business intelligence tools such as Power BI.
What does the beta period mean for DaaS?
We have chosen to release DaaS with a significant but incomplete data surface area to provide early benefit to our customers who need it. The service is fully functional during this period, allowing us the opportunity to shape our future offering by taking the valuable feedback of customers early.
When should I use DaaS instead of APIs or web services?
DaaS provides a deep, more retrospective view of your data (eg. what job types make me the most money?) whereas APIs are better equipped for real-time concerns (eg. where is my vehicle now?).
APIs are geared for software developers looking to extend the functionality of our platform to suit their business. Generally, that will be for process automation and/or integration with third party platforms.
Can I use DaaS to write data back into the platform?
The data we provide via DaaS is read-only. It is focussed on providing fast analytical answers, rather than providing the ability to write data back. Writing of data should be handled by APIs/web services.
Data Availability
What data is available via DaaS?
The data dictionary provides a detailed, up to date listing of the data provided by DaaS.
What about the rest of the data held in the JobWatch platform?
The data we provide will evolve over time as we respond to feedback from customers.
Is the data the same format as provided by web services?
No. The data provided by DaaS is brand-new and purpose-built to allow data professionals to self-service their own analytics requirements.
What format will data be provided in?
We provide flat, read-focused marts that are fast to query and easy to build upon. These human-readable wide tables are designed to prioritise the experience of the data engineer. Most table joins have been de-normalised and common aggregations/calculations are already built-in.
What about deleted data?
The records entirely removed (hard deleted) from the JobWatch platform are not visible in DaaS tables. The records marked as deleted (soft deleted) are provided in DaaS tables with deleted_at column populated with the timestamp of record deletion.
How fresh is the data provided by DaaS?
Please refer to freshness section in the DaaS Reference.
Snowflake
Why do I need a Snowflake account?
Snowflake is the platform we have chosen to use to securely underpin our DaaS service. Your organisation will need an account to be able to access the data we share to you. Creating an account is easy and Snowflake provide a free 30-day trial.
Can I use a Snowflake account I already hold?
It is fine to use an account you already hold - you don't need a dedicated BigChange Snowflake account.
How can I blend DaaS data with data I hold outside of Snowflake?
You can either push the data you hold elsewhere into your Snowflake account, or pull the data from your Snowflake account into your existing database / data lake. There are numerous SaaS platforms that are able to trivialise synchronisation of large quantities of data available today that you might consider for this purpose.
How does Snowflake's pricing model work?
Snowflake's pricing model is unique because it separates compute costs from storage costs, allowing for flexible and scalable usage without large upfront costs.
Here’s a brief overview:
- Storage Costs: Snowflake charges for data storage based on the amount of data stored in Snowflake, including all data in tables, databases, and failed queries. This is measured in terabytes (TB), and pricing is per TB per month. Snowflake uses highly efficient storage techniques to minimize costs.
- Compute Costs: Compute resources, or virtual warehouses, are charged based on their usage. This is measured by the second, with a minimum usage amount per query. You can scale compute resources up or down, and turn them on or off, to manage performance and costs. The cost depends on the size of the virtual warehouse (number of compute nodes) and the region.
- Cloud Services Costs: Snowflake charges for the use of its cloud services layer, which includes the cost of tasks such as queries compilation, optimisation, and encryption. These costs are typically proportional to the compute costs.
- Additional Features Costs: Some additional features and services may incur extra costs, such as data transfer fees when moving data outside of Snowflake's cloud platform, or features like Snowflake’s Materialized Views.
It's important to note that Snowflake offers multiple editions (Standard, Enterprise, Business Critical, and VPS), each with different features and associated costs. Organisations can choose the edition that best fits their needs.
How do I manage my Snowflake billing?
Once you sign up for a Snowflake account, you can manage your billing using the various cost management tools that Snowflake provides. Here, you can view details such as credits consumed, cost per credit, and storage used.
Can I create dashboards on Snowflake?
Yes, Snowflake provides Snowsight Dashboards, a web-based UI for creating visualisations and dashboards directly from SQL queries. Snowsight supports various chart types like bar charts, line charts, scatter plots, etc. for data visualisation. Dashboards can be created by combining multiple visualisations in Snowsight.
However, Snowsight is quite limited in the features that it offers, and we recommend exploring third-party BI tools that integrate with Snowflake and provide better dashboard capabilities.