-
Notifications
You must be signed in to change notification settings - Fork 31
How stuff work
We are using elasticsearch for our datastore. Elasticsearch works like a document datastore that you can connect to with a RESTful API. To create/index new documents in elasticsearch, you do a POST request with a json object. This also means that when GETting a document you will get a json object. All entities in our system are stored as separate documents. The relations are not handled by elasticsearch in any way. The entity simply has the id of the "owner", for example a stream has the id of the resource that it belongs to.
A "flexible and powerful open source, distributed real-time search and analytics engine for the cloud" - http://www.elasticsearch.org/
Elasticsearch is running with the default options. This means you can do http requests to it at port 9200. See Setup for installation instructions.
Configuration of our elasticsearch server can be found here: https://github.com/projectcs13/elasticsearch/blob/configured/config/elasticsearch.yml
Basically, the only thing we have changed is the cluster name.
The entities that exist within our system. Each of these entities is a special type of Elasticsearch document.
Fields currently in a resource:
- user_id
- name
- tags
- description
- type
- manufacturer
- streams
- uri
- polling_freq
- creation_date
A stream is a representation of a stream of data. A stream has meta-data associated with it. The actual data is saved as individual datapoints.
Fields currently in a stream:
- resource_id
- name
- tags
- description
- private
- type
- accuracy
- min_val
- max_val
- quality
- active
- user_ranking
- subscribers
- last_updated
- creation_date
- history_size
- location
Fields currently in a virtual stream:
- name
- description
- tags
- group
- private
- history_size
- last_updated
- creation_date
- function
Fields currently in a datapoint:
- timestamp
- value
- stream_id