Broker vs feed performance

[kikingalgo]

Has anyone experienced slower performance when using CCXT feed with a broker vs using CCXT feed without a broker?

Can anyone explain the difference?

[bartosh]

Just a guess. Can it be caused by this issue?

[sirhc78]

I am definitively getting the same experience during prenext.
11sec per bar to compute few indicators when it run instantaneously without the broker.

I tried to explained my problem here :
#17 (comment)

and here:
https://community.backtrader.com/topic/623/anyone-use-backtrader-to-do-live-trading-on-bitcoin-exchange/274

[sirhc78]

I have found functions which are taking too much time.
Here is my ouput :

time is  2018-10-02 17:07:10.848704  ## start
CCXTBroker getposition 4.0 Seconds
0.01
init pausetrade =  True
***** DATA NOTIF: DELAYED
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:07:25.269656  ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:07:35.509687  ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:07:45.745484  ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:07:55.980936  ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:08:06.219192  ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
***** DATA NOTIF: LIVE
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is  2018-10-02 17:08:20.552561  ## next
***** NEXT: 2018-10-02 15:08:00  6551.0 6551.5 6550.0 6551.5 78848.0 Minute 6
2018-10-02 15:08:00, Close, 6551.50
CCXTBroker getposition 4.0 Seconds
CCXTBroker getposition 4.0 Seconds

It seems that getvalue and getcash functions are taking 2 sec and getposition 4sec.
So I have found my 10-11 sec per bar during prenext.

With profiler, we see that retry_method in CCXTStore take all.

    def retry(method):
        @wraps(method)
        def retry_method(self, *args, **kwargs):
            for i in range(self.retries):
                time.sleep(self.exchange.rateLimit / 1000)
                try:
                    return method(self, *args, **kwargs)
                except (NetworkError, ExchangeError):
                    if i == self.retries - 1:
                        raise
        return retry_method

self.exchange.rateLimit = 2000 so it wait time.sleep(2.0 sec)

Does that make sense to modify this code like that ?

    def retry(method):
        @wraps(method)
        def retry_method(self, *args, **kwargs):
            for i in range(self.retries):
                try:
                    return method(self, *args, **kwargs)
                except (NetworkError, ExchangeError):
                    if i == self.retries - 1:
                        raise
                time.sleep(self.exchange.rateLimit / 1000)
        return retry_method

[bartosh]

Does that make sense to modify this code like that ?

Yes, it makes sense. Please, submit PR. I'll be happy to accept it.

[sirhc78]

May be a wrong PR.

If I load 100 bar and, in prenext, BT request 5 times (2 getcash 3 getvalue) the broker per bar. We will reach this rolling limit of 300 per 5min.

Ideally, we should use a smart solution like this:
https://github.com/tomasbasham/ratelimit

from ratelimit import limits, sleep_and_retry
import datetime
import time
import random


class Test:
    TEN_SEC = 10
    counter = 0


    @sleep_and_retry
    @limits(calls=5, period=TEN_SEC )
    def ratelimiter(self):
        pass
    
    def test(self):
        self.ratelimiter()
        self.counter +=1
        print('test ' + str(self.counter) + ' ' + str(datetime.datetime.now()))
    def test2(self):
        self.ratelimiter()
        self.counter +=1
        print('test2 ' + str(self.counter) + ' ' + str(datetime.datetime.now()))
    
a=Test()
for i in range(1,20) : 
    a.test()
    time.sleep(random.randint(0,2))
    a.test2()

Or what about an option to totally remove getcash/getvalue requests to the broker.
In that case of Bitmex, returned data does not seem relevant as Bitmex permits to trade Futures and not currency pairs.
And it requested 5 times for nothing, and getposition call 2 times retry decorator.

I have never done a PR yet but it will come one day 👍

[VegarS]

I don't think threading or adjusting the rate limiting will solve the main performance issue which is caused by a lack of caching in the broker/store.

For my own testing I have resolved to only updating the balance (cash/value/position) once for each candle, and no more than once per ten seconds. This eliminates the slowness when running on historical data and greatly reduces issues with rate limiting.

My changes can be seen in this commit:
VegarS@179f684

It is not a complete solution, but it is a step in the right direction. A better solution is maybe to periodically update the balances and orders in a separate thread, which will also allow for proper order status notifications. @bartosh: Do periodic updates sound reasonable?