clickhouse-client --database=qoestor --query="system reload dictionaries"
* Check if there are errors in dictionaries clickhouse-client --database=qoestor --query="select * from system.dictionaries"
* Check if a dictionary contains data, for example, for subnets_local_dic clickhouse-client --database=qoestor --query="select * from subnets_local_dic"
In dpiui2, in the **QoE Analytics / Administrator / Dictionary Information** section, you can view a list of dictionaries, errors, and contents. See [[dpi:dpi_components:dpiui:user_guide:qoe_analytics:admin:dics|Dictionary Information]].
===== List of dictionaries =====
^ Dictionary ^ Description ^ Columns ^ Example ^ Notes ^
| **asnum_dic** | Network AS dictionary. Cloud-based, loaded on schedule. Not editable. | 1. AS number\\ 2. AS name | 0 -Reserved AS-, ZZ 1 LVLT-1, US 2 UDEL-DCN, US 3 MIT-GATEWAYS, US 4 ISI-AS, US 5 SYMBOLICS, US 6 BULL-HN, US 7 DSTL, GB 8 RICE-AS, US ... | To manually reload the dictionary, execute:sh /var/qoestor/backend/app_bash/cron_daily.sh |
| **asnum_local_dic**\\ **subnets_local_dic** | Lists your local AS numbers and subnets. Used to determine traffic direction (relevant when DPI is installed on a mirror) and filter subscribers (so host IPs don’t appear in subscriber reports). | **asnum_local_dic**:\\ 1. AS number\\ 2. Display name\\ **subnets_local_dic**:\\ 1. IP or CIDR\\ 2. Name (not displayed in reports, but required by format) | asnum_local_dic:12345 LOCAL 65535 UNKNOWN \\ subnets_local_dic: 192.168.1.0/24 LOCAL 10.64.66.0/24 LOCAL 172.16.0.0 LOCAL 2a02:2168:aaa:bbbb::2 LOCAL | Use parameters SUBSCRIBER_FILTER_MODE and TRAFFIC_DIR_DEF_MODE to control filtering based on these dictionaries. See [[dpi:qoe_analytics:implementation_administration:configuration_setup|Configuration]].\\ :!: Avoid adding very large subnets; split them into smaller ones. Limit – 100000000 |
| **asnum_exclude_dic**\\ **subnets_exclude_dic** | Lists AS numbers and subnets (or individual IPs) to exclude from aggregated logs. Subnets listed here will be ignored when writing to the aggregated log (used for reports). | **asnum_exclude_dic**:\\ 1. AS number\\ 2. Name\\ **subnets_exclude_dic**:\\ 1. IP or CIDR\\ 2. Name | asnum_exclude_dic:12345 LOCAL 65535 LOCAL \\ subnets_exclude_dic: 192.168.1.0/24 LOCAL 10.64.66.0/24 LOCAL 172.16.0.0 LOCAL 2a02:2168:aaa:bbbb::2 LOCAL | Use SUBSCRIBER_EXCLUDE_MODE to control filtering based on these dictionaries. See [[dpi:qoe_analytics:implementation_administration:configuration_setup|Configuration]].\\ :!: Avoid adding very large subnets; split them into smaller ones. Limit – 100000000 |
| **gtp_cells_locations_dic** | Contains a list of the operator’s cellular base stations with coordinates. Used for map-based reporting. | 1. MCC - Mobile Country Code\\ 2. MNC - Mobile Network Code\\ 3. TAC/LAC - Location Area Code\\ 4. CID/SAC/ECI - Cell ID\\ 5. LAT - latitude\\ 6. LON - longitude\\ 7. CELL_ID - cell identifier (operator-defined)\\ 8. CELL_NAME - cell name\\ 9. CELL_DESCRIPTION - cell description | 250 1 17774 4881420 55.783184763200005 37.525521418974996 cell_id_0 cell_name_0 cell_description_0 250 1 17716 3398401 55.86518562276 37.66309891922 cell_id_2 cell_name_2 cell_description_2 250 1 17716 3398411 55.86312193015 37.657007482905 cell_id_4 cell_name_4 cell_description_4 250 1 17782 2359297 55.86370294584 37.661393065359995 cell_id_5 cell_name_5 cell_description_5 250 1 1603 26057 55.80400515858 37.63568625476 cell_id_1 cell_name_1 cell_description_1 250 1 17781 2914571 55.799170861170005 37.6376949577 cell_id_6 cell_name_6 cell_description_6 250 1 1605 23895 55.869888445769995 37.64814730281 cell_id_3 cell_name_3 cell_description_3 | — |
| **net_protocols_dic** | Network protocol dictionary. Not editable, updated with QoE Stor updates. | 1. Port number\\ 2. Protocol name | 0 HOPOPT 1 ICMP 2 IGMP 3 GGP 4 IPv4 5 ST 6 TCP 7 CBT 8 EGP 9 IGP 10 BBN-RCC-MON 11 NVP-II ... | — |
| **protocols_dic**\\ **protocols_groups_dic** | Dictionaries of application protocols and their groups. Cloud-based, loaded on schedule. Not editable. | **protocols_dic**:\\ 1. Port number\\ 2. Protocol name\\ 3. Group ID\\ **protocols_groups_dic**:\\ 1. Group ID\\ 2. Group name | protocols_dic: 1 tcpmux 1 2 compressnet 1 5 rje 1 7 echo 1 11 systat 7 13 daytime 7 17 qotd 7 18 msp 8 19 chargen 7 20 ftp-data 9 21 ftp 9 22 ssh 8 23 telnet 8 25 smtp 5 27 nsw-fe 7 ... \\ protocols_groups_dic: 1 Network services 2 Web browsing 3 P2P 4 Gaming 5 E-Mail 6 Instant messengers | To manually reload dictionaries, execute:sh /var/qoestor/backend/app_bash/cron_daily.sh |
| **settings_dic** | Service dictionary of settings. Generated by scripts based on configuration files. Not editable. | 1. Parameter\\ 2. Value | TRAFFIC_DIR_DEF_MODE 0 SUBSCRIBER_FILTER_MODE 0 SUBSCRIBER_EXCLUDE_MODE 0 URLS_CATEGORIES_DIC_AUTOLOAD_ENABLED 1 ASNUM_DIC_AUTOLOAD_ENABLED 1 ULR_REPLACE_LOGIN_WITH_VCHANNEL 0 ULR_USE_DIC_WHEN_REPLACING_LOGIN 0 ULR_VCHANNEL_NAME_DIC_AUTOLOAD_ENABLED 0 ULR_VCHANNEL_NAME_DIC_URL NAT_AGG_LOG_FIELDS_TO_SAVE_BITMASK 0x654 NAT_IMPORT_FROM_FULLFLOW NAT_AGG_LOG_GROUP_TIME_INTERVAL 15 FULLFLOW_AGG_LOG_FIELDS_TO_SAVE_BITMASK 0x39fff FULLFLOW_AGG_LOG_GROUP_TIME_INTERVAL 15 CLICKSTREAM_AGG_LOG_FIELDS_TO_SAVE_BITMASK 0x30ff CLICKSTREAM_AGG_LOG_GROUP_TIME_INTERVAL 15 CLICKSTREAM_ADD_INFO_FROM_FULLFLOW 1 SUBSCRIBER_BIND_MODE 1 FILL_IP_LOGIN_BINDING_FROM_FULLFLOW 1 NAT_ADD_LOGIN_FROM_IP_LOGIN_BINDING 1 APP_ID qoestor APP_VERSION 1.9.0 | — |
| **subscribers_dic** | Subscriber dictionary. | 1. IP address\\ 2. Login\\ 3. Switch ID\\ 4. Switch port\\ 5. Equipment vendor\\ 6. Cable\\ 7. Contract\\ 8. Services\\ 9. MAC address (reserved for future use) | 10.64.66.100 login 5 port1 unit_vendor cabel contract services mac 10.64.66.101 login 2 port1 unit_vendor cabel contract services mac 10.64.66.102 login 3 port1 unit_vendor cabel contract services mac 10.64.66.103 login 4 port1 unit_vendor cabel contract services mac 10.64.66.104 login 5 port1 unit_vendor cabel contract services mac 10.64.66.105 login 5 port2 unit_vendor cabel contract services mac 10.64.66.106 login 5 port3 unit_vendor cabel contract services mac | — |
| **switches_dic** | Hierarchical dictionary of equipment (access and backbone switches). | 1. Equipment ID UInt64\\ 2. Name\\ 3. Type\\ 4. Region\\ 5. Address\\ 6. Switch IP\\ 7. Operator\\ 8. Flag\\ 9. Parent switch ID\\ 10. Port\\ 11. Owner | 1 Switch 1 Ethernet Region1 Address 1 10.140.1.18 oper1 0 0 2 Switch 2 Ethernet Region2 Address 2 10.140.2.18 oper1 0 0 3 Switch 3 Ethernet Region3 Address 3 10.140.3.18 oper1 0 1 port1 4 Switch 4 Ethernet Region4 Address 4 10.140.4.18 oper1 0 3 port1 5 Switch 5 Ethernet Region5 Address 5 10.140.5.18 oper1 0 4 port1 | — |
| **crc_dic** | Dictionary of CRC errors on switch ports. | 1. Switch ID\\ 2. Port\\ 3. CRC value | 2 port1 450 5 port1 550 5 port2 500 4 port1 780 | — |
| **urlcats_dic**\\ **urlcats_host_dic** | Host category dictionaries. Used to determine host category membership. Cloud-based, loaded on schedule. Not editable. | **urlcats_dic**:\\ 1. Category ID\\ 2. Category code\\ **urlcats_host_dic**:\\ 1. Host\\ 2. Category ID | urlcats_dic: 1 unknown 2 software_tools 3 search_engine 4 0_other 5 school 6 proxys 7 tabak_alkohol 8 religia \\ urlcats_host_dic: iris06-gold-ssl.gameloft.com 1 satfrog-tv.ddns.net:5890 1 vs824.vcdn.biz 1 cs05.trafmag.com 1 | To manually reload dictionaries, execute:sh /var/qoestor/backend/app_bash/cron_daily.sh |
| **ulr_vchannel_name_dic** | Dictionary of virtual channel names. Used to generate statistics by virtual channels. | 1. DPI device ID\\ 2. Virtual channel number (vchannel)\\ 3. Name | 0 1 vchan_0_1 0 2 vchan_0_2 0 3 vchan_0_3 0 4 vchan_0_4 1 1 vchan_1_1 1 2 vchan_1_2 1 3 vchan_1_3 1 4 vchan_1_4 | — |
===== OpenCellID service =====
If coordinate data is missing in the [[dpi:qoe_analytics:implementation_administration:dictionaries#list_of_dictionaries|gtp_cells_locations_dic]] dictionary for some base stations, it is possible to use a **service for enriching the SSG GUI dictionary with base station data from an external source via API**.
OpenCellID is a project that collects GPS coordinates of cell towers and their corresponding location area identifiers. The OpenCellID database is published under the international Creative Commons Attribution-ShareAlike 4.0 license, promoting free use and distribution of data.
The service is activated by adding the following parameters to /var/qoestor/backend/.env:
#Enable autoload of OpenCellID dictionary
CELL_TOWER_AUTOLOAD_ENABLED=1
# 0 - Disabled (default)
# 1 - Enabled
#Token for the LocalAPI platform
CELL_TOWER_LOADER_API_TOKEN=pk.fce9af82655c117ef42c88d36885cc8b
#Logging level
CELL_TOWER_LOG=1
# 0 - Info
# 1 - Debug
#Time interval for updating empty data for geo reports
CELL_TOWER_RESPONSE_HOUR=
#Defined in hours
A token can be obtained via https://my.unwiredlabs.com/trial. After registration, the token will be available:
{{:dpi:qoe_analytics:implementation_administration:dictionaries:screenshot_from_2022-09-02_18-24-48.png?1000|}}
When you click //Show token//, a free key with a limit of 100 requests per day will be displayed.