Configuring Managed Service for ClickHouse for Graphite
Managed Service for ClickHouse can be used as data storage for Graphite
The table engine GraphiteMergeTree
Note
If trimming and aggregation or averaging are not required, any ClickHouse table engine
To configure a database to work with Graphite:
- Prepare the environment.
- Create a cluster.
- Register the rollup configuration in a cluster.
- Create and set up a virtual machine.
- Connect the virtual machine to a database.
- Create a GraphiteMergeTree table.
- Configure Graphite.
If you no longer need the resources you created, delete them.
Prepare the environment
Go to the management console
Create a cluster
- Create a Managed Service for ClickHouse cluster in any suitable configuration with the
db1
database and public access to all of its hosts. Save the database name, username, and password. - In the management console
, select Managed Service for ClickHouse and go to the created cluster. Save the cluster ID in the Overview tab. - In the top-right corner, click Connect. In the Shell tab, save the
--host
parameter from the Sample connection string field, e.g.,rc1a-2sqal8f01znegjkj.mdb.nemax.nebius.cloud
. This is the cluster host FQDN which you will need later.
Register the rollup configuration in a cluster
Register the rollup
configuration in a cluster to trim and aggregate or average the contents of a database for Graphite:
If you don't have the Nebius AI command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
Prepare the yaml file
graphite-rollup.yaml
with the description of therollup
parameters, for example:name: test_rollup patterns: - regexp: click_cost function: max retention: - age: 86400 precision: 60
-
Replace
<CLUSTER_ID>
with your cluster ID and<path to yaml file>
, withgraphite-rollup.yaml
, and then run this command:ncp managed-clickhouse cluster add-graphite-rollup <CLUSTER_ID> --rollup-file-name <path to yaml file>
Where:
<CLUSTER_ID>
: Cluster ID.<path to yaml file>
: Path tographite-rollup.yaml
.
For more information about the
managed-clickhouse cluster add-graphite-rollup
command, see the CLI reference.Note
To delete the
rollup
configuration, use themanaged-clickhouse cluster remove-graphite-rollup
command. For more information about the command, see the CLI reference.
Create and configure a virtual machine
-
In the same cloud network where the cluster is located, create a VM on Linux.
-
Connect to the VM via SSH.
-
Connect the DEB repository
ClickHouse:sudo apt update && sudo apt install -y apt-transport-https ca-certificates dirmngr && \ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4 && \ echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \ /etc/apt/sources.list.d/clickhouse.list
-
Install the dependencies and the
clickhouse-client
client application:sudo apt update && sudo apt install -y clickhouse-client
-
Download the configuration file for
clickhouse-client
:mkdir -p ~/.clickhouse-client && \ wget "https://storage.ai.nebius.cloud/doc-files/clickhouse-client.conf.example" \ --output-document ~/.clickhouse-client/config.xml
-
Get an SSL certificate:
sudo mkdir --parents /usr/local/share/ca-certificates/NebiusAI/ && \ sudo wget "https://storage.nemax.nebius.cloud/certs/CA.pem" \ --output-document /usr/local/share/ca-certificates/NebiusAI/NebiusAICA.crt && \ sudo chmod 655 /usr/local/share/ca-certificates/NebiusAI/NebiusAICA.crt && \ sudo update-ca-certificates
Connect the virtual machine to a database
-
Run the ClickHouse CLI with the following parameters: replace
<host FQDN>
,<database name>
,<database username>
, and<database user password>
with the previously saved values.clickhouse-client --host <host FQDN> \ --secure \ --user <database username> \ --database <database name> \ --password <database user password> \ --port 9440
Create a GraphiteMergeTree table
In the ClickHouse CLI interface, run a query to create a GraphiteMergeTreerollup
section described earlier:
CREATE TABLE GraphiteTable
(
metric String,
time DateTime,
value Int64,
version UInt64
)
ENGINE = GraphiteMergeTree('test_rollup')
PARTITION BY time
ORDER BY cityHash64(version, metric)
Configure Graphite
Configure Graphite to save metrics in the ClickHouse cluster. In this case, data is trimmed automatically by the ClickHouse server according to the parameters you specified.
-
Install the
carbon-clickhouse
utility in the system with Graphite:wget https://github.com/go-graphite/carbon-clickhouse/releases/download/v0.11.2/carbon-clickhouse_0.11.2_amd64.deb && \ sudo apt-get install $(pwd)/carbon-clickhouse_0.11.2_amd64.deb
-
Configure
carbon-clickhouse
. -
Run
carbon-clickhouse
:sudo systemctl enable carbon-clickhouse && \ sudo systemctl start carbon-clickhouse
-
Install the
graphite-clickhouse
utility:wget https://github.com/go-graphite/graphite-clickhouse/releases/download/v0.13.2/graphite-clickhouse_0.13.2_amd64.deb && \ sudo apt-get install $(pwd)/graphite-clickhouse_0.13.2_amd64.deb
-
Configure
graphite-clickhouse
. -
Run
graphite-clickhouse
:sudo systemctl enable graphite-clickhouse && \ sudo systemctl start graphite-clickhouse
For more information about configuring Graphite, see the relevant documentation
.
How to delete the resources you created
Delete the resources you no longer need to avoid being charged for them: