ElasticSearchDB

This class a wrapper around elasticsearch-py. It is used to query Elasticsearch database.

class DIRAC.Core.Utilities.ElasticSearchDB.ElasticSearchDB(host, port, user=None, password=None, indexPrefix='', useSSL=True)

Bases: object

class ElasticSearchDB
Parameters:
  • url (str) – the url to the database for example: el.cern.ch:9200
  • gDebugFile (str) – is used to save the debug information to a file
  • timeout (int) – the default time out to Elasticsearch
  • RESULT_SIZE (int) – The number of data points which will be returned by the query.
RESULT_SIZE = 10000
__init__(host, port, user=None, password=None, indexPrefix='', useSSL=True)

c’tor

Parameters:
  • self – self reference
  • host (str) – name of the database for example: MonitoringDB
  • port (str) – The full name of the database for example: ‘Monitoring/MonitoringDB’
  • user (str) – user name to access the db
  • password (str) – if the db is password protected we need to provide a password
  • indexPrefix (str) – it is the indexPrefix used to get all indexes
  • useSSL (bool) – We can disable using secure connection. By default we use secure connection.
bulk_index(indexprefix, doc_type='_doc', data=None, mapping=None, period=None)
Parameters:
  • indexPrefix (str) – index name.
  • doc_type (str) – the type of the document
  • data (list) – contains a list of dictionary
  • mapping (dict) – the mapping used by elasticsearch
  • period (str) – We can specify which kind of indices will be created. Currently only daily and monthly indexes are supported.
clusterName = ''
createIndex(indexPrefix, mapping, period=None)
Parameters:
  • indexPrefix (str) – it is the index name.
  • mapping (dict) – the configuration of the index.
  • period (str) – We can specify, which kind of index will be created. Currently only daily and monthly indexes are supported.
deleteByQuery(indexName, query)

Delete data by query

Parameters:
  • indexName (str) – the name of the index
  • query (str) – the query that we want to issue the delete on
deleteIndex(indexName)
Parameters:indexName (str) – the name of the index to be deleted…
exists(indexName)

it checks the existance of an index

Parameters:indexName (str) – the name of the index
static generateFullIndexName(indexName, period=None)

Given an index prefix we create the actual index name. Each day an index is created.

Parameters:
  • indexName (str) – it is the name of the index
  • period (str) – We can specify, which kind of indexes will be created. Currently only daily and monthly indexes are supported.
getDocTypes(indexName)
Parameters:indexName (str) – is the name of the index…
Returns:S_OK or S_ERROR
getIndexPrefix()

It returns the DIRAC setup.

getIndexes()

It returns the available indexes…

getUniqueValue(indexName, key, orderBy=False)
Parameters:
  • indexName (str) – the name of the index which will be used for the query
  • orderBy (dict) – it is a dictionary in case we want to order the result {key:’desc’} or {key:’asc’}
Returns:

a list of unique value for a certain key from the dictionary.

index(indexName, doc_type='_doc', body=None, docID=None)
Parameters:
  • indexName (str) – the name of the index to be used
  • doc_type (str) – the type of the document
  • body (dict) – the data which will be indexed (basically the JSON)
  • id (int) – optional document id
Returns:

the index name in case of success.

pingDB()

Try to connect to the database

Returns:S_OK(TRUE/FALSE)
query(index, query)

Executes a query and returns its result (uses ES DSL language).

Parameters:
  • self – self reference
  • index (basestring) – index name
  • query (dict) – It is the query in ElasticSearch DSL language
update(index, doctype='_doc', query=None, updateByQuery=True, id=None)

Executes an update of a document, and returns S_OK/S_ERROR

Parameters:
  • self – self reference
  • index (basestring) – index name
  • doctype (basestring) – type of document
  • query (dict) – It is the query in ElasticSearch DSL language
  • updateByQuery (bool) – A bool to determine updation by update by query or index values using index function.
  • id (int) – ID for the document to be created.