Dr. Yves J. Hilpisch
The Python Quants GmbH
Risk Disclaimer¶
Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits. Leverage can work against you. Due to the certain restrictions imposed by the local law and regulation, German resident retail client(s) could sustain a total loss of deposited funds but are not subject to subsequent payment obligations beyond the deposited funds. Be aware and fully understand all risks associated with the market and trading. Prior to trading any products, carefully consider your financial situation and experience level. Any opinions, news, research, analyses, prices, or other information is provided as general market commentary, and does not constitute investment advice. FXCM & TPQ will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information.
Speaker Disclaimer¶
The speaker is neither an employee, agent nor representative of FXCM and is therefore acting independently. The opinions given are their own, constitute general market commentary, and do not constitute the opinion or advice of FXCM or any form of personal or investment advice. FXCM assumes no responsibility for any loss or damage, including but not limited to, any loss or gain arising out of the direct or indirect use of this or any other content. Trading forex/CFDs on margin carries a high level of risk and may not be suitable for all investors as you could sustain losses in excess of deposits.
Some Basic Imports¶
import time
import numpy as np
import pandas as pd
import datetime as dt
import cufflinks as cf
from pylab import plt
cf.set_config_file(offline=True)
plt.style.use('seaborn')
%matplotlib inline
You can install fxcmpy via
pip install fxcmpy
The documentation is currently found under http://fxcmpy.tpq.io
Read the license & risk warning carefully.
import fxcmpy
fxcmpy.__version__
Retrieving Historical Tick Data¶
Available Symbols¶
from fxcmpy import fxcmpy_tick_data_reader as tdr
print(tdr.get_available_symbols())
Retrieving Tick Data¶
start = dt.datetime(2018, 3, 26)
stop = dt.datetime(2018, 3, 29)
%time td = tdr('EURUSD', start, stop)
type(td)
td.get_raw_data().info()
Working with the Tick Data¶
%time td.get_data().info()
%time td.get_data().info()
sub = td.get_data(start='2018-03-27 12:00:00', end='2018-03-27 16:00:00')
sub.head()
sub['Mid'] = sub.mean(axis=1)
sub['SMA'] = sub['Mid'].rolling(1000).mean()
sub[['Mid', 'SMA']].iplot()
Reading Historical Candles Data¶
Available Symbols¶
from fxcmpy import fxcmpy_candles_data_reader as cdr
print(cdr.get_available_symbols())
Retrieving Candles Data¶
start = dt.datetime(2018, 3, 1)
stop = dt.datetime(2018, 3, 25)
period must be one of m1, H1 or D1
period = 'H1'
candles = cdr('EURUSD', start, stop, period)
type(candles)
data = candles.get_data()
data.info()
data.head()
D1 for period currently only works for time windows before the current year.
start = dt.datetime(2017, 1, 3)
stop = dt.datetime(2017, 12, 31)
period = 'D1'
candles = cdr('EURUSD', start, stop, period)
candles.get_data().info()
Working with Candles Data¶
sub = pd.DataFrame({'Mid': (candles.get_data()['BidClose'] + candles.get_data()['AskClose']) / 2},
index = candles.get_data().index)
sub['Returns'] = np.log(sub / sub.shift(1))
sub.head()
sub.iplot(subplots=True)
sub['Returns'].iplot(kind='histogram')
Quant Fingures¶
data = candles.get_data()[['AskOpen', 'AskHigh', 'AskLow', 'AskClose']]
data.columns = ['open', 'high', 'low', 'close']
data.info()
qf = cf.QuantFig(data, title='EUR/USD', legend='top',
name='EUR/USD', datalegend=False)
qf.iplot()
qf.add_bollinger_bands(periods=10, boll_std=2,
colors=['magenta', 'grey'], fill=True)
qf.data.update()
qf.iplot()
qf.add_rsi(periods=14, showbands=False)
qf.data.update()
qf.iplot()
Backtesting AI-Based Algorithmic Trading Strategy¶
The following example is simplified and for illustration purposes only. Among others, it does not consider transactions costs or bid-ask spreads.
Data Retrieval¶
period = 'H1'
candles = cdr('EURUSD', start, stop, period)
candles.get_data().info()
data = pd.DataFrame(candles.get_data()[['AskClose', 'BidClose']].mean(axis=1),
columns=['midclose'])
data.info()
data.tail()
data.iplot()
Feature Preparation¶
data['returns'] = np.log(data / data.shift(1))
lags = 5
cols = []
for lag in range(1, lags + 1):
col = 'lag_%s' % lag
data[col] = data['returns'].shift(lag)
cols.append(col)
cols
from pylab import plt
plt.style.use('seaborn')
%matplotlib inline
data['direction'] = np.sign(data['returns'])
to_plot = ['midclose', 'returns', 'direction']
data[to_plot].iloc[-75:].plot(figsize=(10, 6),
subplots=True, style=['-', '-', 'ro'], title='EUR/USD');
# the "patterns" = 2 ** lags
np.digitize(data[cols], bins=[0])[:10]
2 ** len(cols)
data.dropna(inplace=True)
Support Vector Machines¶
from sklearn import svm
model = svm.SVC(C=100)
data.info()
%time model.fit(np.sign(data[cols]), np.sign(data['returns']))
Predicting Market Direction¶
In the prediction, a +1 means a positive return is expected and a -1 means a negative return is expected.
pred = model.predict(np.sign(data[cols]))
pred[:15]
Vectorized Backtesting¶
data['position'] = pred
data['strategy'] = data['position'] * data['returns']
# unleveraged | no bid-ask spread or transaction costs | only in-sample
data[['returns', 'strategy']].cumsum().apply(np.exp).iplot()
data['position'].value_counts()
Train Test Split¶
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
Split Feature Sets¶
# mu = data['returns'].mean()
# v = data['returns'].std()
# bins = [mu - 0.5 * v, mu, mu + 0.5 * v]
train_x, test_x, train_y, test_y = train_test_split(
data[cols].apply(lambda x: np.digitize(x, bins=[0])),
np.sign(data['returns']),
test_size=0.50, random_state=111)
train_x.sort_index(inplace=True)
train_y.sort_index(inplace=True)
test_x.sort_index(inplace=True)
test_y.sort_index(inplace=True)
# the patterns = buckets ** lags
train_x.head(5)
test_x.tail(5)
2 ** len(cols)
ax = data['midclose'].iloc[-75:][train_x.index].plot(style=['bo'],
figsize=(10, 6))
data['midclose'].iloc[-75:][test_x.index].plot(style=['ro'], ax=ax)
data['midclose'].iloc[-75:].plot(ax=ax, lw=0.5, style=['k--']);
Model Fitting & Prediction¶
model.fit(train_x, train_y)
train_pred = model.predict(train_x)
accuracy_score(train_y, train_pred)
test_pred = model.predict(test_x)
accuracy_score(test_y, test_pred)
Vectorized Backtesting — Direct Predictions
pred = model.predict(data[cols].apply(lambda x: np.digitize(x, bins=[0])).dropna())
pred[:15]
data['position'] = pred
data['strategy'] = data['position'] * data['returns']
# in-sample | unleveraged | no bid-ask spread or transaction costs
data.loc[train_x.index][['returns', 'strategy']].cumsum().apply(np.exp).iplot()
# out-of-sample | unleveraged | no bid-ask spread or transaction costs
data.loc[test_x.index][['returns', 'strategy']].cumsum().apply(np.exp).iplot()
# number of trades
sum(data['position'].diff() != 0)