Is there any limit on number of securities to get price data at once?

Blueshift
1

Currently i am trying to get price data for about 1800 securities together. The below code works when i have say 100 securities but throwing an error with high number of securities. Any suggestions ?

 

    try:
        price_data = data.history(context.securities, 'close',
            context.params['indicator_lookback']+1,
            context.params['indicator_freq'])
        volume_data = data.history(context.securities, 'volume',
            context.params['indicator_lookback']+1,
            context.params['indicator_freq'])    
    except:
        print("error in price data")
        return

Error

Error in strategy - Traceback (most recent call last):
  File "momentum_and_volatility.py", line 84, in run_strategy
  File "momentum_and_volatility.py", line 96, in generate_signals
  File "blueshift_data/assets/_adjustments.pyx", line 323, in blueshift_data.assets._adjustments.get_adjusted_series
  File "blueshift_data/assets/_adjustments.pyx", line 353, in blueshift_data.assets._adjustments.get_adjusted_series
IndexError: index 0 is out of bounds for axis 0 with size 0
Blueshift Alert[2022-01-03 09:45:00+05:30] ERROR in algorithm:Fatal error, will exit.
Blueshift Alert[2022-10-05 15:34:35.549515+05:30] WARNING :Shutting down Blueshift gracefully.

 

2

The error you are getting means some of the securities returned empty dataframes (i.e. they do not have data for the duration of the query).



Querying 1800 securities at one go using data.history is a very wrong approach. At present we do not have any instrinsic limit on Blueshift, but in the next release, it will be explictly capped at 200 securities. This (200) is a large number in itself to query on every event of the algo. Also, most live brokers support a max number, typically between 30 to 200. So using a number more than 200 will make your strategy essentially non-tradable in live environment, even if you are able to run a backtest.



If you want to query a large number of securities, the right approach is to use the Blueshift Pipeline APIs. Use these to compute factors on a large universe and filter it down to a smaller set (<=200) to use in data.history or data.current. Also, make sure the security is live on the date of query (that will be automatically be ensured if you are using Pipeline APIs for factoring/ filtering first).