Python Trading

The low learning curve Python programming language has grown in popularity over the past decade. Data Scientists, algorithmic developers, quantitative financial professionals, and market enthusiasts have helped this become a strong tool for algorithmic research, development, and trading. Python for the trading industry comes with tools including:

  • Jupyter notebooks
  • NumPy for High-Speed Numerical Processing
  • Pandas for Efficient Data Analysis and Time Series Analysis Techniques
  • Matplotlib for Data Visualization
  • TA-Lib for Technical Analysis
  • Tensor flow

Posts

Yahoo Finance ORCL - Profile Tab

Web Scraping a Stock Symbol’s URL using Yahoo Finance with Python for Alternative Data Links

Alternative Data is growing as a necessary weapon for traders and quantitative investors. Yet there are many barriers to alternative data success.

According to Greenwich Associates, some of the barriers include:

  • Data incompatibility
  • Poor or ineffective Data Sources
  • Human Capital Required

Overcoming Some of these Barriers with Data or Symbology Mapping

Symbology Maps

Every trading firm has some form of symbology map. Most cover the basic use cases for their specific trading. For example, one firm may have maps of Reuters Identification Codes (RICS) to their internally used maps. It is all that firm needs, and therefore that is where most trading firms stop.

Expanding Symbology Maps to Cover the Need of Alternative Data

Data incompatibility for traders and quant researchers begins with understanding the linkage between data sets. The data in the Alternative Dataset most likely is missing the trading symbol that could be used to link the data to the stock market data.

For example, the alternative data may contain a reference to the company name, or to a web site. Consider that your altdata has “www.oracle.com.” The data incompatibility begins with a missing map that takes this URL and maps it to “ORCL,” the trading symbol used by traders. Both the URL and the trading Symbol map to a company name “Oracle Corporation.”

This is a non-traditional Symbology Map.

www.oracle.com = ORCL.

Your alternative data may need other non-standard symbology mappings. Consider some other mappings for example:

  • Corporate Officers
  • Major Vendors
  • Raw Materials
  • Financial Analysts

Symbology maps are needed to overcome AltData barriers. They directly address issues with data incompatibility thereby improving ineffective data sources and decreasing the demand on

Building A URL to Symbology Map

The following Python code can be used to build this non-traditional symbol to URL map. This code (written in Python 3) uses web scraping and basic logic to pull the URL from the Yahoo Finance “profile” page/tab to get the URL for a given list of stock symbols.

Keep in mind – If you use a web service like Yahoo Finance make sure you review their terms of service in collecting data from Yahoo (or any other) site. Be fair. Be Ethical.

Yahoo Finance ORCL - Profile Tab
Yahoo Finance ORCL – Profile Tab

The following code was written using JupyterLab on CloudQuant.ai – the advanced data science platform provided by CloudQuant.

The code loops through a list of symbols [‘SBUX’, ‘MET’, ‘CAT’, ‘JNJ’, ‘ORCL’] and reads the Yahoo finance page to build a map of symbols to company URL.

import bs4 as BeautifulSoup 
from bs4 import SoupStrainer
import re
import urllib.request 
import pandas as pd
import requests
symbols = ['SBUX', 'MET', 'CAT', 'JNJ', 'ORCL']

headers = {'User-agent': 'Mozilla/5.0'}
mySymbols = {}

# looping through all my symbols
for s in symbols:
    vals = {}
    #getting the symbol "profile" from Yahoo finance.
    # The url for the stock appears on this page.
    url = ("https://finance.yahoo.com/quote/{}/profile?p={}".format(s,s))
    webpage = requests.get(url, headers=headers)
    soup = BeautifulSoup.BeautifulSoup(webpage.content) 

    # the title has the company name but has additional information in the format of 
    # (SSS) profile and .... 
    # where SSS is the symbol. We remove this extra title to get the company name.
    title = soup.find("title")
    tmp = title.get_text()
    rxTitle = re.compile(r'\(.*$')
    coName = rxTitle.sub("", tmp)
    
    # looping through all the links in the document.
    # The company web site is the the one that doesn't have yahoo in the reference,
    # and has a blank title.
    for link in soup.find_all('a', href=True):
        try:
            if link['target'] and "" == link['title']:
                m = re.search('yahoo', link['href'], flags=re.IGNORECASE)
                if None == m:
                    
                    url = link['href']
                    webpage = requests.get(url, headers=headers)
                    soup = BeautifulSoup.BeautifulSoup(webpage.content) 
                    
                    vals = {"company":coName, "url":link['href']} 
                    print (s, vals)
                    mySymbols[s] = vals
        except:
            pass

Output

The output from this portion of the script will produce the console prints.

Console Output is OK, but…

Outputting this data to the console is helpful, mainly to easily see what is happening in my code. But what I really want is a Pandas data frame that will allow me to sort, filter, and save the data into a CSV. To put the collected data map into a dataframe I simply use the following code.

#placing my data into a data frame
df = pd.DataFrame.from_dict(mySymbols, orient = 'index')
df

Dataframe Output


Author: Tayloe Draughon, Senior Product Manager, CloudQuant

Numpy Memory Leak Fix

Fixing Python Memory Leaks

A few of our power users reported that long-running backtests would sometimes run out of memory. These power-users are the people who often find new trading strategies and so we wanted to work with them to improve the performance of our backtesting tools. Over the past couple of weeks, our senior engineer found that the problem wasn’t in our code, but in one of the popular Python libraries that we use. We found the problem in numpy and numba. 
Algo Trading powered by Alternative Data Sets

Conversations: Effects of Alternative Data Sets on Trading Algorithms

,

What effect can alternative data sets have on trading algorithms?

We asked a few of our teammates and systematic traders what the effect of alternative data sets is on trading algos. We thought we could spread some insight as to why our alternative data is so valuable the also developers. We all start using the smallest and most basic data sets, such as the generic S&P500 stock prices. This is all just fine until you want to take your development to the next level. You can delve deeper into predicting the rise and fall of each stock using these specified alternative data sets. CloudQuant offers a wide and expanding variety of these data sets for free, giving our users easy access to alternative data in order to help them improve their trading strategies.
The teammates featured are:
  • Tayloe Draughon- Sr. Product Manager
  • James Chang- Quantitative Portfolio Manager
  • Morgan Slade- CEO
  • Steve Pettinato- Portfolio Manager
  • Paul Tunney- Client Success Manager
Trading strategies require thought

Conversations: Recommendations to Someone Starting out at CloudQuant

, ,
The CloudQuant team discusses their helpful thoughts for beginners on CloudQuant. We want to boost everyone starting out on our platform in their algo development and backtesting. Everyone in our company uses the CloudQuant website and coding platform in one way or another. We all use our own application, just like the crowd researchers. When we say that our free backtesting tools are “institutional grade” we really mean it. Every algo we run in our trading and investment strategies is proven in the same backtesting engine as the crowd uses. We rely on the scorecards, the reports, and the simulated trades to ensure that our trading is successful.
Quantamental alternative data

Conversations: Learning Python within CloudQuant

What was your experience like learning Python within CloudQuant?

We asked our portfolio managers and product management teammates who code in Python to explain their starting experiences in programming with Python with CloudQuant. We wanted to share with everyone what encouraged them to keep learning throughout the years. Everyone here codes as part of their job. This includes the CEO all the way down to the interns. We rely on our Backtesting Engine to ensure that trading algorithms work well before committing money to the automated trading strategies. But we also use JupyterLab in our daily work. We generate our reports, monitor our systems, and do all sorts of tasks in Python. Python has overtaken the spreadsheet in CloudQuant. The teammates featured in this video are:
  • Morgan Slade- CEO
  • Tayloe Draughon- Senior Product Manager
  • James Chang- Quantitative Portfolio Manager
  • Rob Ferguson- Quantitative Equity Portfolio Manager
  • Simon Zhang- Quantitative Analyst
Watch this video to gain insight as to what got our amazing coders to where they are.
backtest chart

Conversations: What we wish we knew when we started AlgoTrading

, ,
CloudQuant’s portfolio managers and quantitative algo traders look back on their starts in Algorithmic Trading. This candid overview allows everyone to see the “Things We Wish We Knew When We Started AlgoTrading”.  This is a short collection of the interviews with some of our amazing coders here in the office
Trevor Trinkino Quantitative Trader

Machine Learning FXCM Webinar with Trevor Trinkino of CloudQuant – Part 2/3

On May 15th Trevor Trinkino presented part two of a three-part Machine Learning webinar with FXCM. Part one is here.

Part 2  – Preprocess data for Random Forest. PnL and prediciton improvements…

In part two Trevor goes over how to clean and pre-process data from CloudQuant to use in a Random Forest Classifier. He then looks at the prediction and PnL improvements seen from utilizing this classifier as well as adjusting the hyper-parameters of the model. Files mentioned in the video are available from our Google Drive below.. Ipython Notebook Trades.csv FilteredClfFeatsBF100_50.csv MLClassData.csv    
Backtest Homepage showing P&L and Sharpe

CloudQuant rolls out Upgrades to Free Stock Market Backtesting System

,
CloudQuant, the trading strategy incubator, announces upgrades to our free stock market backtesting system. The web application allows any market enthusiasts to develop a trading strategy using easy to learn Python programming. Anyone who has ever written a spreadsheet macro or a simple program can easily use the system.

Market Turmoil Generates Opportunity for Proprietary Traders

,
In these times of market turmoil and volatility, the Kershner Trading Group stands ready to provide traders with a firm built on a strong foundation of significant capital investment, innovation-focused trading technology and decades of experience in the active and proprietary trading space. Kershner Trading is actively seeking experienced US Equities Traders
Trevor Trinkino Quantitative Trader

FXCM Machine Learning with Trevor Trinkino

On February 8th Trevor Trinkino presented Machine Learning with FXCM in a webinar. During this presentation, he promised to make available his machine learning Python Notebook and the supporting data file. These are available on our Google drive at: