VAS Experts documentation VAS Experts documentation-mobile
in

Settings

  • Show pagesource
  • Old revisions
  • Export to PDF
  • ODT export
  • Backlinks
  • Show pagesource
  • Old revisions
  • Export to PDF
  • ODT export
  • Backlinks
    •  Stingray Service Gateway by VAS ExpertsQoE AnalyticsImplementation and administrationDictionaries

    Table of Contents

    Dictionaries

    Dictionary configuration

    All dictionaries are located in the folder /var/qoestor/backend/etc/db/ and have the .txt extension.

    Each dictionary has a sample.txt example file that can be used as a template.

    All columns in the dictionaries are separated by a tab character (\t). The number of \t characters must be one less than the number of columns in the dictionary. Pay close attention to this.

    When files are modified, the data is automatically reloaded into the database.

    Some useful commands for working with dictionaries:

    • Speed up dictionary data refresh 
      clickhouse-client --database=qoestor --query="system reload dictionaries"
    • Check if there are errors in dictionaries 
      clickhouse-client --database=qoestor --query="select * from system.dictionaries"
    • Check if a dictionary contains data, for example, for subnets_local_dic 
      clickhouse-client --database=qoestor --query="select * from subnets_local_dic"

    In dpiui2, in the QoE Analytics / Administrator / Dictionary Information section, you can view a list of dictionaries, errors, and contents. See Dictionary Information.

    List of dictionaries

    Dictionary Description Columns Example Notes
    asnum_dic Network AS dictionary. Cloud-based, loaded on schedule. Not editable. 1. AS number
    2. AS name 
    0	-Reserved AS-, ZZ 1	LVLT-1, US 2	UDEL-DCN, US 3	MIT-GATEWAYS, US 4	ISI-AS, US 5	SYMBOLICS, US 6	BULL-HN, US 7	DSTL, GB 8	RICE-AS, US ...
    		 To manually reload the dictionary, execute:
    sh /var/qoestor/backend/app_bash/cron_daily.sh
    asnum_local_dic
    subnets_local_dic     		Lists your local AS numbers and subnets. Used to determine traffic direction (relevant when DPI is installed on a mirror) and filter subscribers (so host IPs don’t appear in subscriber reports). asnum_local_dic:
    1. AS number
    2. Display name
    subnets_local_dic:
    1. IP or CIDR
    2. Name (not displayed in reports, but required by format)     		asnum_local_dic:
    12345	LOCAL 65535	UNKNOWN


    subnets_local_dic: 

    192.168.1.0/24	LOCAL 10.64.66.0/24	LOCAL 172.16.0.0	LOCAL 2a02:2168:aaa:bbbb::2	LOCAL
    		 Use parameters SUBSCRIBER_FILTER_MODE and TRAFFIC_DIR_DEF_MODE to control filtering based on these dictionaries. See Configuration.
    :!: Avoid adding very large subnets; split them into smaller ones. Limit – 100000000
    asnum_exclude_dic
    subnets_exclude_dic     		Lists AS numbers and subnets (or individual IPs) to exclude from aggregated logs. Subnets listed here will be ignored when writing to the aggregated log (used for reports). asnum_exclude_dic:
    1. AS number
    2. Name
    subnets_exclude_dic:
    1. IP or CIDR
    2. Name     		asnum_exclude_dic:
    12345	LOCAL 65535	LOCAL


    subnets_exclude_dic: 

    192.168.1.0/24	LOCAL 10.64.66.0/24	LOCAL 172.16.0.0	LOCAL 2a02:2168:aaa:bbbb::2	LOCAL
    		 Use SUBSCRIBER_EXCLUDE_MODE to control filtering based on these dictionaries. See Configuration.
    :!: Avoid adding very large subnets; split them into smaller ones. Limit – 100000000
    gtp_cells_locations_dic Contains a list of the operator’s cellular base stations with coordinates. Used for map-based reporting. 1. MCC - Mobile Country Code
    2. MNC - Mobile Network Code
    3. TAC/LAC - Location Area Code
    4. CID/SAC/ECI - Cell ID
    5. LAT - latitude
    6. LON - longitude
    7. CELL_ID - cell identifier (operator-defined)
    8. CELL_NAME - cell name
    9. CELL_DESCRIPTION - cell description 
    250	1	17774	4881420	55.783184763200005	37.525521418974996	cell_id_0	cell_name_0	cell_description_0 250	1	17716	3398401	55.86518562276	37.66309891922	cell_id_2	cell_name_2	cell_description_2 250	1	17716	3398411	55.86312193015	37.657007482905	cell_id_4	cell_name_4	cell_description_4 250	1	17782	2359297	55.86370294584	37.661393065359995	cell_id_5	cell_name_5	cell_description_5 250	1	1603	26057	55.80400515858	37.63568625476	cell_id_1	cell_name_1	cell_description_1 250	1	17781	2914571	55.799170861170005	37.6376949577	cell_id_6	cell_name_6	cell_description_6 250	1	1605	23895	55.869888445769995	37.64814730281	cell_id_3	cell_name_3	cell_description_3
    		 —
    net_protocols_dic Network protocol dictionary. Not editable, updated with QoE Stor updates. 1. Port number
    2. Protocol name 
    0	HOPOPT 1	ICMP 2	IGMP 3	GGP 4	IPv4 5	ST 6	TCP 7	CBT 8	EGP 9	IGP 10	BBN-RCC-MON 11	NVP-II ...
    		 —
    protocols_dic
    protocols_groups_dic     		Dictionaries of application protocols and their groups. Cloud-based, loaded on schedule. Not editable. protocols_dic:
    1. Port number
    2. Protocol name
    3. Group ID
    protocols_groups_dic:
    1. Group ID
    2. Group name     		protocols_dic: 
    1	tcpmux	1 2	compressnet	1 5	rje	1 7	echo	1 11	systat	7 13	daytime	7 17	qotd	7 18	msp	8 19	chargen	7 20	ftp-data	9 21	ftp	9 22	ssh	8 23	telnet	8 25	smtp	5 27	nsw-fe	7 ...


    protocols_groups_dic: 

    1	Network services 2	Web browsing 3	P2P 4	Gaming 5	E-Mail 6	Instant messengers
    		 To manually reload dictionaries, execute:
    sh /var/qoestor/backend/app_bash/cron_daily.sh
    settings_dic Service dictionary of settings. Generated by scripts based on configuration files. Not editable. 1. Parameter
    2. Value 
     TRAFFIC_DIR_DEF_MODE	0 SUBSCRIBER_FILTER_MODE	0 SUBSCRIBER_EXCLUDE_MODE	0 URLS_CATEGORIES_DIC_AUTOLOAD_ENABLED	1 ASNUM_DIC_AUTOLOAD_ENABLED	1 ULR_REPLACE_LOGIN_WITH_VCHANNEL	0 ULR_USE_DIC_WHEN_REPLACING_LOGIN	0 ULR_VCHANNEL_NAME_DIC_AUTOLOAD_ENABLED	0 ULR_VCHANNEL_NAME_DIC_URL	 NAT_AGG_LOG_FIELDS_TO_SAVE_BITMASK	0x654 NAT_IMPORT_FROM_FULLFLOW	 NAT_AGG_LOG_GROUP_TIME_INTERVAL	15 FULLFLOW_AGG_LOG_FIELDS_TO_SAVE_BITMASK	0x39fff FULLFLOW_AGG_LOG_GROUP_TIME_INTERVAL	15 CLICKSTREAM_AGG_LOG_FIELDS_TO_SAVE_BITMASK	0x30ff CLICKSTREAM_AGG_LOG_GROUP_TIME_INTERVAL	15 CLICKSTREAM_ADD_INFO_FROM_FULLFLOW	1 SUBSCRIBER_BIND_MODE	1 FILL_IP_LOGIN_BINDING_FROM_FULLFLOW	1 NAT_ADD_LOGIN_FROM_IP_LOGIN_BINDING	1 APP_ID	qoestor APP_VERSION	1.9.0
    		 —
    subscribers_dic Subscriber dictionary. 1. IP address
    2. Login
    3. Switch ID
    4. Switch port
    5. Equipment vendor
    6. Cable
    7. Contract
    8. Services
    9. MAC address (reserved for future use) 
    10.64.66.100	login	5	port1	unit_vendor	cabel	contract	services	mac 10.64.66.101	login	2	port1	unit_vendor	cabel	contract	services	mac 10.64.66.102	login	3	port1	unit_vendor	cabel	contract	services	mac 10.64.66.103	login	4	port1	unit_vendor	cabel	contract	services	mac 10.64.66.104	login	5	port1	unit_vendor	cabel	contract	services	mac 10.64.66.105	login	5	port2	unit_vendor	cabel	contract	services	mac 10.64.66.106	login	5	port3	unit_vendor	cabel	contract	services	mac
    		 —
    switches_dic Hierarchical dictionary of equipment (access and backbone switches). 1. Equipment ID UInt64
    2. Name
    3. Type
    4. Region
    5. Address
    6. Switch IP
    7. Operator
    8. Flag
    9. Parent switch ID
    10. Port
    11. Owner 
    1	Switch 1	Ethernet	Region1	Address 1	10.140.1.18	oper1	0	0		 2	Switch 2	Ethernet	Region2	Address 2	10.140.2.18	oper1	0	0		 3	Switch 3	Ethernet	Region3	Address 3	10.140.3.18	oper1	0	1	port1	 4	Switch 4	Ethernet	Region4	Address 4	10.140.4.18	oper1	0	3	port1	 5	Switch 5	Ethernet	Region5	Address 5	10.140.5.18	oper1	0	4	port1
    		—
    crc_dic Dictionary of CRC errors on switch ports. 1. Switch ID
    2. Port
    3. CRC value 
    2	port1	450 5	port1	550 5	port2	500 4	port1	780
    		 —
    urlcats_dic
    urlcats_host_dic     		Host category dictionaries. Used to determine host category membership. Cloud-based, loaded on schedule. Not editable. urlcats_dic:
    1. Category ID
    2. Category code
    urlcats_host_dic:
    1. Host
    2. Category ID     		urlcats_dic: 
    1	unknown 2	software_tools 3	search_engine 4	0_other 5	school 6	proxys 7	tabak_alkohol 8	religia


    urlcats_host_dic: 

    iris06-gold-ssl.gameloft.com    1 satfrog-tv.ddns.net:5890        1 vs824.vcdn.biz  1 cs05.trafmag.com        1
    		 To manually reload dictionaries, execute:
    sh /var/qoestor/backend/app_bash/cron_daily.sh
    ulr_vchannel_name_dic Dictionary of virtual channel names. Used to generate statistics by virtual channels. 1. DPI device ID
    2. Virtual channel number (vchannel)
    3. Name 
    0	1	vchan_0_1 0	2	vchan_0_2 0	3	vchan_0_3 0	4	vchan_0_4 1	1	vchan_1_1 1	2	vchan_1_2 1	3	vchan_1_3 1	4	vchan_1_4
    		 —

    OpenCellID service

    If coordinate data is missing in the gtp_cells_locations_dic dictionary for some base stations, it is possible to use a service for enriching the SSG GUI dictionary with base station data from an external source via API.

    OpenCellID is a project that collects GPS coordinates of cell towers and their corresponding location area identifiers. The OpenCellID database is published under the international Creative Commons Attribution-ShareAlike 4.0 license, promoting free use and distribution of data.

    The service is activated by adding the following parameters to 

    /var/qoestor/backend/.env

    : 

    #Enable autoload of OpenCellID dictionary
CELL_TOWER_AUTOLOAD_ENABLED=1
# 0 - Disabled (default)
# 1 - Enabled
#Token for the LocalAPI platform
CELL_TOWER_LOADER_API_TOKEN=pk.fce9af82655c117ef42c88d36885cc8b
#Logging level
CELL_TOWER_LOG=1
# 0 - Info
# 1 - Debug
#Time interval for updating empty data for geo reports
CELL_TOWER_RESPONSE_HOUR=
#Defined in hours

    A token can be obtained via https://my.unwiredlabs.com/trial. After registration, the token will be available:

    When you click Show token, a free key with a limit of 100 requests per day will be displayed.

    Was this information helpful?