Skip to content

docs(DWH): feature branch for DWH #4766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
62 changes: 62 additions & 0 deletions menu/navigation.json
Original file line number Diff line number Diff line change
@@ -3013,6 +3013,68 @@
],
"label": "Distributed Data Lab",
"slug": "data-lab"
},
{
"items": [
{
"label": "Overview",
"slug": "../data-warehouse"
},
{
"label": "Concepts",
"slug": "concepts"
},
{
"label": "Quickstart",
"slug": "quickstart"
},
{
"label": "FAQ",
"slug": "faq"
},
{
"items": [
{
"label": "Create a Data Warehouse deployment",
"slug": "create-data-warehouse-deployment"
},
{
"label": "Update a Data Warehouse deployment",
"slug": "update-data-warehouse-deployment"
},
{
"label": "Delete a Data Warehouse deployment",
"slug": "delete-data-warehouse-deployment"
},
{
"label": "Connect applications to a deployment",
"slug": "connect-applications"
},
{
"label": "Connect your deployment to BI tools",
"slug": "connect-bi-tools"
}
],
"label": "How to",
"slug": "how-to"
},
{
"items": [
{
"label": "Known differences with regular ClickHouse®",
"slug": "known-differences"
},
{
"label": "Example datasets",
"slug": "example-datasets"
}
],
"label": "Additional content",
"slug": "reference-content"
}
],
"label": "Data Warehouse",
"slug": "data-warehouse"
}
],
"label": "Managed Services",
61 changes: 61 additions & 0 deletions pages/data-warehouse/concepts.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
meta:
title: Data Warehouse for ClickHouse® - Concepts
description: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
content:
h1: Data Warehouse for ClickHouse® - Concepts
paragraph: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
tags: data warehouse clickhouse concepts glossary terms definitions
dates:
published: 2025-05-07
validation: 2025-05-07
categories:
- data-warehouse
---

## Autoscaling

Autoscaling refers to the ability of a Data Warehouse for ClickHouse® to automatically adjust the number of instances without manual intervention.
Scaling mechanisms ensure that resources are provisioned dynamically to handle incoming requests efficiently while minimizing idle capacity and cost.

## ClickHouse®

ClickHouse® is a high-performance, column-oriented, distributed database management system designed for real-time analytics. It is optimized for handling large volumes of data with fast query performance, making it ideal for applications requiring up-to-date insights. ClickHouse stores data in a columnar format, which reduces I/O operations and speeds up query execution. It supports distributed processing across multiple nodes, enabling horizontal scaling and fault tolerance through replication. ClickHouse provides a powerful SQL interface and offers advanced features like real-time data ingestion, compression, and indexing, making it a robust solution for analytical workloads.


## Column-oriented storage

ClickHouse® stores data in a column-oriented format, which significantly optimizes read performance for analytical queries. By storing data in columns rather than rows, ClickHouse reduces the amount of I/O operations needed during query execution, as it only reads the necessary columns from disk.

## Compression

ClickHouse® uses advanced compression algorithms to reduce storage requirements and improve query performance by minimizing data transfer. Compression not only helps in saving disk space but also accelerates data retrieval and processing by reducing the amount of data that needs to be read from storage and transferred over the network.

## Distributed processing

ClickHouse® supports distributed processing across multiple nodes, allowing it to handle extremely large datasets efficiently and scale horizontally. This architecture enables ClickHouse to distribute data and queries across a cluster, improving performance and reliability by leveraging the combined resources of all nodes.

## Horizontal scaling

Horizontal scaling refers to the process of adding more nodes to the cluster to increase its capacity and performance. This approach allows the cluster to handle larger datasets and higher query loads by distributing the data and processing tasks across additional nodes. Data Warehouse for ClickHouse® deployments [scale automatically](#autoscaling) according to the incoming workload.

## Indexing

ClickHouse employs various indexing techniques, such as primary key and skip indexes, to speed up query execution and data retrieval. The primary key index allows for efficient point lookups and range queries, while skip indexes help in quickly skipping over large chunks of data that do not match query conditions, thus reducing the overall query time.

## Node

In the context of a distributed Data Warehouse for ClickHouse® cluster, a node refers to an individual instance that stores and processes a portion of the data. Each node participates in data distribution, query execution, and replication to ensure balanced load, fault tolerance, and high availability. Nodes communicate with each other to coordinate tasks, execute queries in parallel, and maintain synchronized data replicas. They are configured with specific settings to define their roles and manage resources, allowing the cluster to scale and perform efficiently.

## Replica set

A replica set consists of multiple nodes that store identical copies of the same data. This setup ensures fault tolerance and high availability by providing redundancy. If one node in the replica set fails, another node can take over, ensuring continuous data access and processing. ClickHouse® automatically handles data replication and failover, making it a reliable solution for mission-critical applications.

## SQL support

ClickHouse® provides a powerful SQL interface, enabling users to perform complex queries and data manipulations using familiar SQL syntax. This extensive SQL support includes a wide range of functions and features, such as subqueries, window functions, and user-defined functions, making it accessible to both analysts and developers.


## Vertical scaling

Vertical scaling refers to the process of increasing the resources of individual nodes within the cluster. Vertical scaling enhances the performance and capacity of individual nodes, allowing them to handle larger datasets and more complex queries more efficiently. Vertical scaling is often used in conjunction with [horizontal scaling](#horizontal-scaling) to optimize performance and resource utilization in a Data Warehouse for ClickHouse® deployment.
15 changes: 15 additions & 0 deletions pages/data-warehouse/faq.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
meta:
title: Data Warehouse for ClickHouse® FAQ
description: Discover Scaleway Data Warehouse for ClickHouse® and find answers to general questions.
content:
h1: Data Warehouse for ClickHouse®
dates:
validation: 2025-05-07
category: managed-services
productIcon: DataWarehouseProductIcon
---

## What Data Warehouse for ClickHouse®?

Its like a warehouse but for data, and its for clickhouse also
43 changes: 43 additions & 0 deletions pages/data-warehouse/how-to/connect-applications.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
meta:
title: How to connect applications to your deployment
description: Learn how to connect your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
content:
h1: How to Connect Applications to Your Deployment
paragraph: Learn how to connect your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
tags: connect applications deployment data warehouse clickhouse
dates:
validation: 2025-05-07
posted: 2025-05-07
categories:
- data-warehouse
---

This page explains how to connect your applications to your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com).

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

1. Click **Data Warehouse** under **Managed Services** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the Data Warehouse deployment you want to connect to an application. The overview tab of the deployment displays.

3. Click the **Actions** button in the top-right corner of the page. A drop-down menu displays.

4. Select ¨**Connect using frameworks**. The connection wizard displays.

<Message type="note">
To connect your deployment with BI tools, refer to the [dedicated documentation](/data-warehouse/how-to/connect-bi-tools/).
</Message>

5. Click the **Download** link to download the required TLS certificate.

6. Select your preferred framework:
- Protocols: Select the appropriate protocol, then run the displayed command in a terminal. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.
- Languages: Select the desired language, then add the code displayed to your application code. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.

Your application is now connected to your Data Warehouse for ClickHouse® deployment using the administrator account.
14 changes: 14 additions & 0 deletions pages/data-warehouse/how-to/connect-bi-tools.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
meta:
title: How to Connect Your Deployment to BI Tools
description: Learn how to connect your Scaleway Data Warehouse for ClickHouse® deployment to Business Intelligence (BI) tools.
content:
h1: How to Connect Your Deployment to BI Tools
paragraph: Learn how to connect your Scaleway Data Warehouse for ClickHouse® deployment to Business Intelligence (BI) tools.
tags: connect bi-tools deployment data warehouse clickhouse
dates:
validation: 2025-05-07
posted: 2025-05-07
categories:
- data-warehouse
---
54 changes: 54 additions & 0 deletions pages/data-warehouse/how-to/create-deployment.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
meta:
title: How to create a Data Warehouse for ClickHouse® deployment
description: This page provides a guide on how to set up a Data Warehouse using ClickHouse® using the Scaleway console.
content:
h1: How to create a Data Warehouse for ClickHouse® deployment
paragraph: This page provides a guide on how to set up a Data Warehouse using ClickHouse® using the Scaleway console.
tags: clickhouse data-warehouse deployment
dates:
validation: 2025-05-13
posted: 2025-05-07
categories:
- data-warehouse
---

A Data Warehouse is a centralized repository designed to store, manage, and analyze large volumes of structured and semi-structured data. When powered by ClickHouse®, it provides performance for real-time analytics and complex queries. This guide will walk you through the steps to deploy a Data Warehouse for ClickHouse® deployment using the Scaleway console.

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

## Creating a Data Warehouse Deployment

1. Click **ClickHouse®** under **Managed Services** on the side menu. The Managed Data Warehouse for ClickHouse® page displays.

3. Click **+ Create deployment**. The Data Warehouse deployment creation wizard displays.

4. Select the **region** in which to create your deployment.

5. Choose an autoscaling configuration:
- **Presets**: select one of several pre-configured deployments adapted to the most common use cases.
- **Custom**: manually enter the vCPU autoscaling minimum and maximum values, and the number of nodes for your deployment.
<Message type="note">
You can update the autoscaling configuration of your deployment at any time after its creation.
</Message>

6. Enter a name for your deployment, or keep the automatically generated one.

7. Configure the administrator credentials for your deployment by entering a password, or by generating a random one.
<Message type="note">
The username cannot be modified.
</Message>

8. Click the **+ Advanced settings** button, then configure the following elements:

- Select a ClickHouse® version.
- Customize your deployment's storage according to your needs.
- Select a network configuration.

9. Optionally, you can use the cost estimator to simulate the cost of your deployment.

10. Click **Create deployment** to confirm. A list of your Data Warehouse for ClickHouse® deployments displays, showing the newly created resources.
8 changes: 8 additions & 0 deletions pages/data-warehouse/how-to/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
meta:
title: Data Warehouse - How Tos
description: Practical guides for using Scaleway Data Warehouse.
content:
h1: Data Warehouse - How Tos
paragraph: Practical guides for using Scaleway Data Warehouse.
---
60 changes: 60 additions & 0 deletions pages/data-warehouse/how-to/manage-delete-deployment.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
meta:
title: How to manage a Data Warehouse for ClickHouse® deployment
description: This guide shows you how to manage a Data Warehouse for ClickHouse® deployment using the Scaleway console.
content:
h1: How to manage a Data Warehouse for ClickHouse® deployment
paragraph: This guide shows you how to manage a Data Warehouse for ClickHouse® deployment using the Scaleway console.
tags: clickhouse data-warehouse deployment management
dates:
validation: 2025-05-13
posted: 2024-02-20
categories:
- data-warehouse
---

This page explains how to manage and delete your Data Warehouse deployment.

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage a Data Warehouse deployment

1. Click **Data Warehouse** under **Managed Services** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the Data Warehouse deployment you want to manage. The overview tab of the deployment displays. From this view, you can see the configuration of your deployment.

3. Click the **Settings** tab.

4. From this view, you can:

- Add, consult, or delete tags for your deployment.
- [Delete your Data Warehouse deployment](/data-warehouse/how-to/manage-delete-deployment/#how-to-delete-a-data-warehouse).

<Message type="note">
Once you have created a Data Warehouse deployment, you cannot edit certain parameters, such as the node type, or its computing resources. You must [create a new Data Warehouse deployment](/data-warehouse/how-to/create-depoyment/) instead.
</Message>

## How to delete a Data Warehouse deployment

<Message type="important">
This action is irreversible and will permanently delete this Data Warehouse deployment and all its associated data.
</Message>

1. Click **Data Warehouse** under **Managed Services** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the Data Warehouse deployment you want to delete. The **Overview** tab of the deployment displays.

3. Click the **Settings** tab, then select **Delete deployment**.

4. Enter **DELETE** in the confirmation pop-up to confirm your action.

5. Click **Delete Data Warehouse deployment**.

<Message type="note">
You can also delete a Data Warehouse deployment by clicking <Icon name="more" /> next to its name, then **Delete**.
</Message>
60 changes: 60 additions & 0 deletions pages/data-warehouse/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
meta:
title: Data Warehouse Documentation
description: Dive into Scaleway Data Warehouse with our quickstart guides, how-tos, tutorials and more.
---

<ProductHeader
productName=" Data Warehouse"
productLogo="dataWarehouse"
description=" Data Warehouse is designed to help users in their reporting and data analysis tasks for business intelligence."
url="/data-warehouse/quickstart"
label=" Data Warehouse Quickstart"
/>

## Getting Started

<Grid>
<SummaryCard
title="Quickstart"
icon="rocket"
description="Learn how to create, use, manage, and delete a Data Warehouse in a few steps."
label="View Quickstart"
url="/data-warehouse/quickstart/"
/>
<SummaryCard
title="Concepts"
icon="info"
description="Core concepts that give you a better understanding of Scaleway Data Warehouse."
label="View Concepts"
url="/data-warehouse/concepts/"
/>
<SummaryCard
title="How-tos"
icon="help-circle-outline"
description="Check our guides to creating, using, and managing Data Warehouses and their features."
label="View How-tos"
url="/data-warehouse/how-to/"
/>
<SummaryCard
title="How-tos"
icon="help-circle-outline"
description="Check our guides to creating, using, and managing Data Warehouses and their features."
label="View How-tos"
url="/data-warehouse/how-to/"
/>
<SummaryCard
title="Additional Content"
icon="book-open-outline"
description="Go further with detailed, in-depth information on Data Warehouse for ClickHouse®."
label="View Additional Content"
url="/data-warehouse/reference-content/"
/>
</Grid>

## Changelog

<ChangelogList
productName="data-warehouse"
numberOfChanges={3}
/>
Loading
Oops, something went wrong.
Loading
Oops, something went wrong.