Profile

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Sharding configuration ======
{{indexmenu_n>7}}

QoE Stor can work in sharding mode: horizontal cluster scaling, in which parts of the same database are placed on different shards (nodes).
Through sharding, you can:
1. Distribute the load on the IPFIX inserts from several DPIs, spreading it over the nodes.
2. At times speed up the construction of reports by increasing the machine resources-nodes: the construction of reports is performed in parallel and glued into the final report.

===== 1 Create a cluster =====

To create a QoE Stor cluster, you must specify the cluster node settings on each node in the file /etc/clickhouse-server/config.d/qoestor-cluster-config.xml

Edit file on 1st node

<code>
<?xml version="1.0"?>
<yandex>
    <remote_servers>
        <qoestor_cluster>
            <shard>
                <replica>
                    <host>127.0.0.1</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>127.0.0.2</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>127.0.0.3</host>
                    <port>9000</port>
                </replica>
            </shard>
        </qoestor_cluster>
    </remote_servers>
</yandex>
</code>

For each shard, you must specify the host IP or its name. Save your changes.

Copy the file to the rest of the cluster nodes.

Restart the database with the <code>fastor-db-restart</code> command on each node. The cluster is ready to run.

<note important>Port 9000 must be open on all firewall cluster nodes in the internal network between the cluster nodes.</note>


===== 2 Adding a cluster to the GUI =====

==== 1 Master node connection ====

Choose which node you will use as the master. Enable it in the GUI settings. Enable cluster mode. See screenshot below.

{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-master-node.png?direct&400 |}}

<note important>Port 8123 must be open on all cluster nodes.</note>

==== 2 Management via SSH ====

In the GUI Administrator / Hardware section, add all nodes for SSH control.

{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-ssh.png?direct&400 |}}

==== 3 Setting up IPFIX receivers ====

In the QoE Stor configuration, configure the receivers for each node.

{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-receivers.png?direct&400 |}}

Reconfigure DPI to send IPFIX to the appropriate QoE cluster nodes.

<note important>Distribute the load evenly. One or more DPIs must send IPFIX to one QoE Stor node.</note>  

<note warning>Do not send IPFIX from a single DPI to multiple QoE Stor nodes.</note>

xxxxxxxxxx
 
====== Sharding configuration ======{{indexmenu_n>7}}​QoE Stor can work in sharding mode: horizontal cluster scaling, in which parts of the same database are placed on different shards (nodes).Through sharding, you can:1. Distribute the load on the IPFIX inserts from several DPIs, spreading it over the nodes.2. At times speed up the construction of reports by increasing the machine resources-nodes: the construction of reports is performed in parallel and glued into the final report.​===== 1 Create a cluster =====​To create a QoE Stor cluster, you must specify the cluster node settings on each node in the file /etc/clickhouse-server/config.d/qoestor-cluster-config.xml​Edit file on 1st node​<code><?xml version="1.0"?><yandex>    <remote_servers>        <qoestor_cluster>            <shard>                <replica>                    <host>127.0.0.1</host>                    <port>9000</port>                </replica>            </shard>            <shard>                <replica>                    <host>127.0.0.2</host>