Dr. Yves J. Hilpisch
The Python Quants GmbH
Risk Disclaimer¶
Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits. Leverage can work against you. Due to the certain restrictions imposed by the local law and regulation, German resident retail client(s) could sustain a total loss of deposited funds but are not subject to subsequent payment obligations beyond the deposited funds. Be aware and fully understand all risks associated with the market and trading. Prior to trading any products, carefully consider your financial situation and experience level. Any opinions, news, research, analyses, prices, or other information is provided as general market commentary, and does not constitute investment advice. FXCM & TPQ will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information.
Speaker Disclaimer¶
The speaker is neither an employee, agent nor representative of FXCM and is therefore acting independently. The opinions given are their own, constitute general market commentary, and do not constitute the opinion or advice of FXCM or any form of personal or investment advice. FXCM assumes no responsibility for any loss or damage, including but not limited to, any loss or gain arising out of the direct or indirect use of this or any other content. Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits.
Reading FXCM Tick Data¶
import pandas as pd
import datetime as dt
import cufflinks as cf # Cufflinks
cf.set_config_file(offline=True) # set the plotting mode to offline
The Tick Reader Class¶
from fxcm_tick_reader import fxcm_tick_reader
Available Symbols¶
print(fxcm_tick_reader.get_available_symbols())
Retrieving Tick Data¶
start = dt.datetime(2018, 3, 7)
stop = dt.datetime(2018, 3, 8)
%time td = fxcm_tick_reader('EURUSD', start, stop)
type(td)
td.get_raw_data().info()
td.get_raw_data().tail(10)
Working with the Tick Data¶
%time td.get_data().info()
%time td.get_data().info()
%%time
data = td.get_data(start='2018-03-06 08:00:00', end='2018-03-06 16:00:00')
data.info()
data['Bid'].iplot()
Connecting to the FXCM RESTful API¶
You can install fxcmpy via
pip install fxcmpy
The documentation is currently found under http://fxcmpy.tpq.io
import fxcmpy
api = fxcmpy.fxcmpy(config_file='fxcm.cfg')
instruments = api.get_instruments()
print(instruments)
Historical Data¶
Retrieving Historical Data¶
candles = api.get_candles('USD/JPY', period='D1', number=10)
# 10 most recent days | daily
candles
start = dt.datetime(2017, 1, 1)
end = dt.datetime(2018, 1, 1)
candles = api.get_candles('USD/JPY', period='D1',
start=start, stop=end)
candles['askclose'].iplot()
The parameter period must be one of m1, m5, m15, m30, H1, H2, H3, H4, H6, H8, D1, W1 or M1.
# 50 most recent 1 minute bars
candles = api.get_candles('EUR/USD', period='m1', number=150)
candles.tail(10)
Visualization of the Data¶
data = candles[['askopen', 'askhigh', 'asklow', 'askclose']]
data.columns = ['open', 'high', 'low', 'close']
data.info()
qf = cf.QuantFig(data, title='EUR/USD', legend='top',
name='EUR/USD', datalegend=False)
qf.iplot()
qf.add_bollinger_bands(periods=10, boll_std=2,
colors=['magenta', 'grey'], fill=True)
qf.data.update()
qf.iplot()
Using Machine Learning for Market Prediction¶
The following example is simplified and for illustration purposes only. Among others, it does not consider transactions costs or bid-ask spreads.
import datetime
import numpy as np
import pandas as pd
Data Retrieval¶
candles = api.get_candles('EUR/USD', period='m5',
start=dt.datetime(2018, 3, 7),
stop=dt.datetime(2018, 3, 9))
data = pd.DataFrame(candles[['askclose', 'bidclose']].mean(axis=1),
columns=['midclose'])
data.info()
data.tail()
data.iplot()
Feature Preparation¶
data['returns'] = np.log(data / data.shift(1))
lags = 3
cols = []
for lag in range(1, lags + 1):
col = 'lag_%s' % lag
data[col] = data['returns'].shift(lag)
cols.append(col)
col = 'momentum'
data[col] = data['returns'].rolling(5).mean().shift(1)
cols.append(col)
cols
from pylab import plt
plt.style.use('seaborn')
%matplotlib inline
data['direction'] = np.sign(data['returns'])
to_plot = ['midclose', 'returns', 'direction']
data[to_plot].iloc[:100].plot(figsize=(10, 6),
subplots=True, style=['-', '-', 'ro'], title='EUR/USD');
# the "patterns" = 2 ** lags
np.digitize(data[cols], bins=[0])[:10]
2 ** len(cols)
data.dropna(inplace=True)
Support Vector Machines¶
from sklearn import svm
model = svm.SVC(C=100)
data.info()
%time model.fit(np.sign(data[cols]), np.sign(data['returns']))
Predicting Market Direction¶
pred = model.predict(np.sign(data[cols]))
pred[:15]
Vectorized Backtesting¶
data['position'] = pred
data['strategy'] = data['position'] * data['returns']
# unleveraged | no bid-ask spread or transaction costs | only in-sample
data[['returns', 'strategy']].cumsum().apply(np.exp).iplot()
data['position'].value_counts()
Train Test Split¶
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
Split Feature Sets¶
nos = 50
mu = data['returns'].rolling(nos).mean()
v = data['returns'].rolling(nos).std()
def digits(s):
r = np.where(s < mu - v, 0, np.nan)
r = np.where(s > mu - v, 1, r)
r = np.where(s > mu, 2, r)
r = np.where(s > mu + v, 3, r)
return r
data[cols].apply(digits)
bins = [mu - v, mu, mu + v]
train_x, test_x, train_y, test_y = train_test_split(
data[cols].apply(digits).dropna(),
np.sign(data['returns']).iloc[nos-1:],
test_size=0.50, random_state=111)
train_x.sort_index(inplace=True)
train_y.sort_index(inplace=True)
test_x.sort_index(inplace=True)
test_y.sort_index(inplace=True)
# the patterns = buckets ** lags
train_x.head(5)
test_x.tail(5)
4 ** len(cols)
ax = data['midclose'][train_x.index].plot(style=['b.'], figsize=(10, 6))
data['midclose'][test_x.index].plot(style=['r.'], ax=ax)
data['midclose'].plot(ax=ax, lw=0.5, style=['k--']);
ax = data['midclose'].iloc[-100:][train_x.index].plot(style=['bo'], figsize=(10, 6))
data['midclose'].iloc[-100:][test_x.index].plot(style=['ro'], ax=ax)
data['midclose'].iloc[-100:].plot(ax=ax, lw=0.5, style=['k--']);
Model Fitting & Prediction¶
model.fit(train_x, train_y)
train_pred = model.predict(train_x)
accuracy_score(train_y, train_pred)
test_pred = model.predict(test_x)
accuracy_score(test_y, test_pred)
Vectorized Backtesting — Direct Predictions
pred = model.predict(data[cols].apply(digits).dropna())
pred[:15]
data['position'] = 0.0
data['position'].iloc[nos-1:] = pred
data['strategy'] = data['position'] * data['returns']
# in-sample | unleveraged | no bid-ask spread or transaction costs
data.loc[train_x.index][['returns', 'strategy']].cumsum().apply(np.exp).iplot()
# out-of-sample | unleveraged | no bid-ask spread or transaction costs
data.loc[test_x.index][['returns', 'strategy']].cumsum().apply(np.exp).iplot()
# number of trades
sum(data['position'].diff() != 0)
Retrieving Streaming Data¶
def output(data, dataframe):
print('%3d | %s | %s | %6.4f, %6.4f, %6.4f, %6.4f'
% (len(dataframe), data['Symbol'],
pd.to_datetime(int(data['Updated']), unit='ms'),
data['Rates'][0], data['Rates'][1],
data['Rates'][2], data['Rates'][3]))
api.subscribe_market_data('EUR/USD', (output,))
api.get_last_price('EUR/USD')
api.unsubscribe_market_data('EUR/USD')
Placing Orders via the RESTful API¶
api.get_open_positions()
order = api.create_market_buy_order('EUR/USD', 100)
order.get_currency()
order.get_isBuy()
cols = ['tradeId', 'amountK', 'currency', 'grossPL', 'isBuy']
api.get_open_positions()[cols]
order = api.create_market_buy_order('USD/JPY', 50)
api.get_open_positions()[cols]
order = api.create_market_sell_order('EUR/USD', 25)
order = api.create_market_buy_order('USD/JPY', 50)
api.get_open_positions()[cols]
order = api.create_market_sell_order('EUR/USD', 50)
api.get_open_positions()[cols]
api.close_all_for_symbol('USD/JPY')
api.get_open_positions()[cols]
api.close_all()
api.get_open_positions()
A Time-Based, Automated Algorithm¶
import time
for _ in range(5):
print(50 * '=')
print('TRADE NO {}'.format(_))
order = api.create_market_buy_order('EUR/USD', 100)
time.sleep(1)
print('POSITIONS\n', api.get_open_positions()[cols])
time.sleep(7)
api.close_all_for_symbol('EUR/USD')
print('POSITIONS\n', api.get_open_positions())
time.sleep(7)