Posted by In the first part of the series we installed Elasticsearch. Now its time to fill that Elasticsearch instance with some data.
As mentioned before we are going to store FX buy / sell data in elastic. Before doing this we need to determine a index and a type of our data. For this example I use “banking” as my index and “fx” as type. In this example we also use curl to post JSON data to elastic.
Adding data to Elasticsearch using Curl
The format looks like this. As you can see you can add the index and type to the uri and give a id. Use a unique id, otherwise the entry will be overwritten.
|
1 |
curl -XPOST -H "Content-Type: application/json" 'http://localhost:9200/banking/fx/1' -d '{ "type": "fx", "ticker": "xauusd", "buy": 1.23, "target": 1.25, "stop": 1.1, "way": "long"}' |
Check the entry with the following query.
|
1 |
curl -X GET 'http://localhost:9200/banking/fx/1?pretty' |
Adding data to Elasticsearch using Python
Python is a great scripting language to bulk import data to your Elastic instance. Scripting against Elasticsearch with python is made very easy with the already scripted Python module Elasticsearch. The module is basically a REST api parser. Very basic example script is added below.
Create a json file wich you want to import in Elastic
|
1 |
sudo nano /home/rverhe/signal.json |
Install the python module requirements
|
1 2 3 |
pip3 install elasticsearch Pip3 install simplejson Pip3 install requests |
The example script
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import requests, json from elasticsearch import Elasticsearch from elasticsearch import helpers from elasticsearch.serializer import JSONSerializer import os,sys directory = '/home/rverhe/signal.json' es = Elasticsearch([{'host': 'localhost', 'port':9200 }]) with open(directory, 'r', encoding='utf-8') as f: data=json.loads(f.read()) es.index(index='banking', doc_type='fx', body=data) |