QuantRocket logo
Disclaimer

FX Business Day Strategy › Part 1: Data Collection

Data Collection¶

To backtest the FX strategy, we will collect 1-hour bars from Interactive Brokers for EUR.USD.

EUR.USD is included in QuantRocket's list of free sample data. No QuantRocket subscription is required to collect EUR.USD data but an IBKR account is required. (IBKR provides FX data for free to account holders.)

First, start IB Gateway:

In [1]:
from quantrocket.ibg import start_gateways
start_gateways(wait=True)
Out[1]:
{'ibg1': {'status': 'running'}}

Collect EUR.USD listing¶

The first step is to collect the securities master record for EUR.USD. EUR.USD is included in QuantRocket's free sample data, so we can collect the contract details by specifying the "FREE" country:

In [2]:
from quantrocket.master import collect_ibkr_listings
collect_ibkr_listings(countries="FREE")

{'status': 'success', 'msg': 'successfully loaded IBKR FREE securities'}

Lookup EUR.USD¶

We need to look up the Sid (security ID) for EUR.USD so we can collect historical data for it. We use the command line interface because less typing is required. (You can also look this up in the Data Browser.)

Prefixing a line with ! allows running terminal commands from inside a notebook.

In [3]:
!quantrocket master get --symbols 'EUR.USD' --json | json2yaml

---
  - 
    Sid: "FXEURUSD"
    Symbol: "EUR.USD"
    Exchange: "IDEALPRO"
    Country: null
    Currency: "USD"
    SecType: "CASH"
    Etf: 0
    Timezone: "America/New_York"
    Name: "European Monetary Union Euro"
    PriceMagnifier: 1
    Multiplier: 1
    Delisted: 0
    DateDelisted: null
    LastTradeDate: null
    RolloverDate: null

Collect historical data¶

Next, we create a database for collecting 1-hour EUR.USD bars. Since FX is an OTC market, historical data does not represent actual trades but rather the bid-ask midpoint; we specify MIDPOINT as the bar type to be explicit, but this would be implied for CASH securities even if we omitted the bar_type parameter. We use EUR.USD's nickname, Fiber, to name the database.

The shard parameter is required for intraday databases and determines how to split up large databases into smaller pieces for better performance. Since our universe contains only one pair, we can turn this feature off. See the usage guide to learn more.

In [4]:
from quantrocket.history import create_ibkr_db
create_ibkr_db("fiber-1h", sids="FXEURUSD", bar_size="1 hour", bar_type="MIDPOINT", shard="off")
Out[4]:
{'status': 'successfully created quantrocket.v2.history.fiber-1h.sqlite'}

Then collect the data:

In [5]:
from quantrocket.history import collect_history
collect_history("fiber-1h")
Out[5]:
{'status': 'the historical data will be collected asynchronously'}

Monitor flightlog for completion:

quantrocket.history: INFO [fiber-1h] Collecting history from IBKR for 1 securities in fiber-1h
quantrocket.history: INFO [fiber-1h] Saved 85778 total records for 1 total securities to quantrocket.v2.history.fiber-1h.sqlite

For more detailed feedback during the data collection, check the detailed logs (quantrocket flightlog stream -d):

quantrocket-history-1|Issuing to ibg1 historical data request for 20 D of 1 hour MIDPOINT for EUR.USD CASH (sid FXEURUSD) ending 20070204 17:00:00 UTC
quantrocket-history-1|Issuing to ibg1 historical data request for 20 D of 1 hour MIDPOINT for EUR.USD CASH (sid FXEURUSD) ending 20070224 17:00:00 UTC
...

Next Up¶

Part 2: Time-of-Day Research