Perfect For Beginners!
Free Trading Education!
Free Demo Account!
Good Choice For Experienced Traders!
Stock Market Analysis – How To Use Digital Options With Technical Analysis
Get instant access to exclusive investing ideas, stock lists and powerful research tools with 30 days of MarketBeat All Access for just $1.00.
Market Data and Calendars
Looking for new stock ideas? Want to see which stocks are moving? View our full suite of financial calendars and market data tables, all for free.
Investing Education and Resources
Receive a free world-class investing education from MarketBeat. Learn about financial terms, types of investments, trading strategies and more.
MarketBeat empowers individual investors to make better trading decisions by providing real-time financial data and objective market analysis. Whether you’re looking for analyst ratings, corporate buybacks, dividends, earnings, economic reports, financials, insider trades, IPOs, SEC filings or stock splits, MarketBeat has the objective information you need to analyze any stock. Learn more.
© American Consumer News, LLC dba MarketBeat® 2020. All rights reserved.
326 E 8th St #108, Sioux Falls, SD 57103 | [email protected] | (844) 978-6257
MarketBeat does not provide financial advice and does not issue recommendations or offers to buy stock or sell any security. Learn more.
Curtis Miller’s Personal Website
Curtis Miller’s personal website, with resume, portfolio, blog, etc.
Subscribe Via RSS
Jobs for R Users
Jobs for Python Users
Around September of 2020 I wrote two articles on using Python for accessing, visualizing, and evaluating trading strategies (see part 1 and part 2). These have been my most popular posts, up until I published my article on learning programming languages (featuring my dad’s story as a programmer), and has been translated into both Russian (which used to be on backtest.ru at a link that now appears to no longer work) and Chinese (here and here). R has excellent packages for analyzing stock data, so I feel there should be a “translation” of the post for using R for stock data analysis.
This post is the first in a two-part series on stock data analysis using R, based on a lecture I gave on the subject for MATH 3900 (Data Science) at the University of Utah. In these posts, I will discuss basics such as obtaining the data from Yahoo! Finance using pandas, visualizing stock data, moving averages, developing a moving-average crossover strategy, backtesting, and benchmarking. The final post will include practice problems. This first post discusses topics up to introducing moving averages.
NOTE: The information in this post is of a general nature containing information and opinions from the author’s perspective. None of the content of this post should be considered financial advice. Furthermore, any code written here is provided without any form of guarantee. Individuals who choose to use it do so at their own risk.
Advanced mathematics and statistics have been present in finance for some time. Prior to the 1980s, banking and finance were well-known for being “boring”; investment banking was distinct from commercial banking and the primary role of the industry was handling “simple” (at least in comparison to today) financial instruments, such as loans. Deregulation under the Regan administration, coupled with an influx of mathematical talent, transformed the industry from the “boring” business of banking to what it is today, and since then, finance has joined the other sciences as a motivation for mathematical research and advancement. For example one of the biggest recent achievements of mathematics was the derivation of the Black-Scholes formula, which facilitated the pricing of stock options (a contract giving the holder the right to purchase or sell a stock at a particular price to the issuer of the option). That said, bad statistical models, including the Black-Scholes formula, hold part of the blame for the 2008 financial crisis.
Perfect For Beginners!
Free Trading Education!
Free Demo Account!
Good Choice For Experienced Traders!
In recent years, computer science has joined advanced mathematics in revolutionizing finance and trading, the practice of buying and selling of financial assets for the purpose of making a profit. In recent years, trading has become dominated by computers; algorithms are responsible for making rapid split-second trading decisions faster than humans could make (so rapidly, the speed at which light travels is a limitation when designing systems). Additionally, machine learning and data mining techniques are growing in popularity in the financial sector, and likely will continue to do so. In fact, a large part of algorithmic trading is high-frequency trading (HFT). While algorithms may outperform humans, the technology is still new and playing an increasing role in a famously turbulent, high-stakes arena. HFT was responsible for phenomena such as the 2020 flash crash and a 2020 flash crash prompted by a hacked Associated Press tweet about an attack on the White House.
My articles, however, will not be about how to crash the stock market with bad mathematical models or trading algorithms. Instead, I intend to provide you with basic tools for handling and analyzing stock market data with R. We will be using stock data as a first exposure to time series data, which is data considered dependent on the time it was observed (other examples of time series include temperature data, demand for energy on a power grid, Internet server load, and many, many others). I will also discuss moving averages, how to construct trading strategies using moving averages, how to formulate exit strategies upon entering a position, and how to evaluate a strategy with backtesting.
DISCLAIMER: THIS IS NOT FINANCIAL ADVICE. Furthermore, I have ZERO experience as a trader (a lot of this knowledge comes from a one-semester course on stock trading I took at Salt Lake Community College)! This is purely introductory knowledge, not enough to make a living trading stocks. People can and do lose money trading stocks, and you do so at your own risk!
Getting and Visualizing Stock Data
Getting Data from Yahoo! Finance with quantmod
Before we analyze stock data, we need to get it into some workable format. Stock data can be obtained from Yahoo! Finance, Google Finance, or a number of other sources, and the quantmod package provides easy access to Yahoo! Finance and Google Finance data, along with other sources. In fact, quantmod provides a number of useful features for financial modelling, and we will be seeing those features throughout these articles. In this lecture, we will get our data from Yahoo! Finance.
Let’s briefly discuss this. getSymbols() created in the global environment an object called AAPL (named automatically after the ticker symbol of the security retrieved) that is of the xts class (which is also a zoo -class object). xts objects (provided in the xts package) are seen as improved versions of the ts object for storing time series data. They allow for time-based indexing and provide custom attributes, along with allowing multiple (presumably related) time series with the same time index to be stored in the same object. (Here is a vignette describing xts objects.) The different series are the columns of the object, with the name of the associated security (here, AAPL) being prefixed to the corresponding series.
Yahoo! Finance provides six series with each security. Open is the price of the stock at the beginning of the trading day (it need not be the closing price of the previous trading day), high is the highest price of the stock on that trading day, low the lowest price of the stock on that trading day, and close the price of the stock at closing time. Volume indicates how many stocks were traded. Adjusted close (abreviated as “adjusted” by getSymbols() ) is the closing price of the stock that adjusts the price of the stock for corporate actions. While stock prices are considered to be set mostly by traders, stock splits (when the company makes each extant stock worth two and halves the price) and dividends (payout of company profits per share) also affect the price of a stock and should be accounted for.
Visualizing Stock Data
Now that we have stock data we would like to visualize it. I first use base R plotting to visualize the series.
A linechart is fine, but there are at least four variables involved for each date (open, high, low, and close), and we would like to have some visual way to see all four variables that does not require plotting four separate lines. Financial data is often plotted with a Japanese candlestick plot, so named because it was first created by 18th century Japanese rice traders. Use the function candleChart() from quantmod to create such a chart.
With a candlestick chart, a black candlestick indicates a day where the closing price was higher than the open (a gain), while a red candlestick indicates a day where the open was higher than the close (a loss). The wicks indicate the high and the low, and the body the open and close (hue is used to determine which end of the body is the open and which the close). Candlestick charts are popular in finance and some strategies in technical analysis use them to make trading decisions, depending on the shape, color, and position of the candles. I will not cover such strategies today.
(Notice that the volume is tracked as a bar chart on the lower pane as well, with the same colors as the corresponding candlesticks. Some traders like to see how many shares are being traded; this can be important in trading.)
We may wish to plot multiple financial instruments together; we may want to compare stocks, compare them to the market, or look at other securities such as exchange-traded funds (ETFs). Later, we will also want to see how to plot a financial instrument against some indicator, like a moving average. For this you would rather use a line chart than a candlestick chart. (How would you plot multiple candlestick charts on top of one another without cluttering the chart?)
Below, I get stock data for some other tech companies and plot their adjusted close together.
What’s wrong with this chart? While absolute price is important (pricey stocks are difficult to purchase, which affects not only their volatility but your ability to trade that stock), when trading, we are more concerned about the relative change of an asset rather than its absolute price. Google’s stocks are much more expensive than Apple’s or Microsoft’s, and this difference makes Apple’s and Microsoft’s stocks appear much less volatile than they truly are (that is, their price appears to not deviate much).
One solution would be to use two different scales when plotting the data; one scale will be used by Apple and Microsoft stocks, and the other by Google.
Not only is this solution difficult to implement well, it is seen as a bad visualization method; it can lead to confusion and misinterpretation, and cannot be read easily.
A “better” solution, though, would be to plot the information we actually want: the stock’s returns. This involves transforming the data into something more useful for our purposes. There are multiple transformations we could apply.
One transformation would be to consider the stock’s return since the beginning of the period of interest. In other words, we plot:
This will require transforming the data in the stocks object, which I do next.
This is a much more useful plot. We can now see how profitable each stock was since the beginning of the period. Furthermore, we see that these stocks are highly correlated; they generally move in the same direction, a fact that was difficult to see in the other charts.
Alternatively, we could plot the change of each stock per day. One way to do so would be to plot the percentage increase of a stock when comparing day to day , with the formula:
But change could be thought of differently as:
These formulas are not the same and can lead to differing conclusions, but there is another way to model the growth of a stock: with log differences.
(Here, is the natural log, and our definition does not depend as strongly on whether we use or .) The advantage of using log differences is that this difference can be interpreted as the percentage change in a stock but does not depend on the denominator of a fraction.
We can obtain and plot the log differences of the data in stocks as follows:
Which transformation do you prefer? Looking at returns since the beginning of the period make the overall trend of the securities in question much more apparent. Changes between days, though, are what more advanced methods actually consider when modelling the behavior of a stock. so they should not be ignored.
Charts are very useful. In fact, some traders base their strategies almost entirely off charts (these are the “technicians”, since trading strategies based off finding patterns in charts is a part of the trading doctrine known as technical analysis). Let’s now consider how we can find trends in stocks.
A -day moving average is, for a series and a point in time , the average of the past days: that is, if denotes a moving average process, then:
Moving averages smooth a series and helps identify trends. The larger is, the less responsive a moving average process is to short-term fluctuations in the series . The idea is that moving average processes help identify trends from “noise”. Fast moving averages have smaller and more closely follow the stock, while slow moving averages have larger , resulting in them responding less to the fluctuations of the stock and being more stable.
quantmod allows for easily adding moving averages to charts, via the addSMA() function.
Notice how late the rolling average begins. It cannot be computed until 20 days have passed. This limitation becomes more severe for longer moving averages. Because I would like to be able to compute 200-day moving averages, I’m going to extend out how much AAPL data we have. That said, we will still largely focus on 2020.
You will notice that a moving average is much smoother than the actual stock data. Additionally, it’s a stubborn indicator; a stock needs to be above or below the moving average line in order for the line to change direction. Thus, crossing a moving average signals a possible change in trend, and should draw attention.
Traders are usually interested in multiple moving averages, such as the 20-day, 50-day, and 200-day moving averages. It’s easy to examine multiple moving averages at once.
The 20-day moving average is the most sensitive to local changes, and the 200-day moving average the least. Here, the 200-day moving average indicates an overall bearish trend: the stock is trending downward over time. The 20-day moving average is at times bearish and at other times bullish, where a positive swing is expected. You can also see that the crossing of moving average lines indicate changes in trend. These crossings are what we can use as trading signals, or indications that a financial security is changing direction and a profitable trade might be made.
Visit next week to read about how to design and test a trading strategy using moving averages.
Update: There were some errors in the original code, which were created when I posted this document to WordPress. They should be fixed now.
Charting the stock markets: does technical analysis work?
Maybe it’s best to ignore the analysts and instead stick to a buy-and-hold approach
What’s going to happen? Technical analysts try to predict future price trends by studying past price action and charts
Some investors aren’t interested in spending hours studying a company’s fundamentals. They look for answers by studying price action and charts. Is there something to technical analysis, or is this age-old practice little more than financial astrology?
Technical analysts try to predict future price trends by studying past price action and charts. Different practitioners use different methods, although a trend-trading approach is common.
Investors who rely on fundamental analysis might sell a stock because it appears too expensive, whereas technicians will tend to hold on to the position as long as it continues to rise. Some traders use strict technical trading rules, others take a discretionary approach. Moving averages, technical indicators that measure if a stock is overbought or oversold, trading volumes, chart patterns, measures of market sentiment – these and other tools are used by the technical community.
Technicians often say price has memory, and keep a close eye on so-called support and resistance levels. For example, the Euro Stoxx 600 index is currently just below the 400 level, which marked major market tops in 2000, 2007 and 2020. The 400 level would be seen as resistance, a point likely to be met by increased selling.
If the index decisively breaks above 400, technicians would see that as bullish. While they might not want to buy the index at 390, they would be perfectly happy to buy at 410. Similarly, many technicians turned bullish in 2020, when the Dow Jones index hit ll-time highs for the first time in 13 years, clearing the 14,000 level that marked major tops in 2000 and 2007.
Academics largely see technical analysis as pseudoscientific nonsense. Stock prices are random, says efficient market theorist Burton Malkiel, author of the classic A Random Walk on Wall Street. Investors who rely on technical analysis “will accomplish nothing but increasing substantially the brokerage charges they pay”, he writes.
Legendary investors such as Warren Buffett and Peter Lynch agree. Buffett has said he “realised that technical analysis didn’t work when I turned the chart upside down and didn’t get a different answer”. To Lynch, charts “are great for predicting the past”.
The practice remains popular among investors, however. Research indicates up to 40 per cent of foreign exchange traders see technical analysis as important for predicting price action over short time horizons. About one-third of equity fund managers utilise technical analysis, according to a major 2020 survey.
Though technical analysis is usually associated with shorter time frames, some long-term investors who eschew speculative activity use technical analysis to supplement their fundamental investment theses. Rory Gillen, co-founder of Dublin-based Merrion Capital and now running GillenMarkets.com, has argued that technical analysis helps assess market psychology and that some medium-term technical indicators are “useful tools in the investment bag”.
Though a “fundamentally-oriented person”, Josh Brown of Ritholtz Wealth Management “can’t imagine doing without charts in some way, shape or form”. Similarly, legendary investor Anthony Bolton, often dubbed Britain’s answer to Warren Buffett, says combining fundamental analysis with chart analysis is better than using fundamental analysis on its own.
“If I were on a desert island and allowed just one investment tool,” Bolton once said, “it would be the chart.”
The continued prominence of technical analysis in financial circles was highlighted earlier this year by a squabble between Bill Gross and Jeffrey Gundlach, the world’s two most prominent bond managers. The multidecade bond bull market would be over, Gross declared, if 10-year US government bonds exceeded 2.6 per cent “because it’s so strong and so important in terms of technical analysis”. Not so, said Gundlach: “The last line in the sand is 3 per cent on the 10-year. That will define the end of the bond bull market from a classic-chart perspective, not 2.6.”
While technical analysis remains widely used, that doesn’t mean it’s not bunkum. Indeed, many technical traders would be the first to accept that the field is full of charlatans. As bond expert and author Martin Fridson has written: “The only thing we know for certain about technical analysis is that it’s possible to make a living publishing a newsletter on the subject.”
Such newsletters are full of references to obscure Japanese candlestick chart patterns, Elliott Wave theory, Fibonacci numbers, and all kinds of other vague and unverified assertions.
The same can be said of fundamental analysis, however. Billionaire investor Mark Cuban once scoffed that “fundamentals is a word invented by sellers to find buyers. metrics created to help stockbrokers sell stocks, and to give buyers reassurance when buying stocks”.
That may be cynical, but it’s a documented fact that most fundamental managers underperform the markets. Most investors would be better adopting a buy-and-hold approach rather than painstakingly studying stockbroker notes in a futile attempt to gain an edge.
What do the studies say? Academics are generally very sceptical of technical analysis. One study that analysed the performance of technical trading rules did find some moving average strategies outperformed between 1934 and 1986 but they have since stopped working. David Aronson, author of Evidence-Based Technical Analysis, tested the performance of 6,402 technical rules; none outperformed the S&P 500.
However, some studies have yielded more positive results, including research conducted by Andrew Lo, a highly-regarded quantitative expert at MIT. Additionally, a mountain of academic research confirms momentum strategies outperform, confirming what technical traders have always preached – that “prices are never too high to begin buying or too low to begin selling”, as 1920s Wall Street legend Jesse Livermore once put it.
Others argue that technicians often combine technical rules, and that this is not accounted for in studies. Similarly, the best strategies are likely to be tightly-held secrets. Accordingly, they suggest the best approach is to analyse the performance of investors who use technical analysis.
Here, the evidence is mixed. One study that examined the portfolios of more than 10,000 fund managers over a 20-year period found that funds that made use of technical analysis “appear to have provided a meaningful advantage to their investors” compared to those who relied solely on fundamentals.
Among ordinary investors, however, the picture is very different. One study that examined the trading records of Dutch investors over a six-year period found those who used technical analysis were much more likely to overtrade and to speculate, the net result being they underperformed investors who didn’t use technical analysis by 8.4 percentage points annually.
For most people, the best approach is to ignore the analysts – both technical and fundamental – and instead stick to a buy-and-hold approach. Investors seeking to outperform, on the other hand, can always combine the two approaches, à la Anthony Bolton, while being mindful to steer clear of technical approaches that encourage excessive trading.
How can they be married? Bolton, now retired, would take a bigger position if the technicals confirmed his fundamental views; if the technicals deteriorated, he reviewed his investment thesis to see if he was missing something.
If he discovered a new interesting company, the first thing he wanted to know was if others had already recognised what he was just discovering, and the price chart would “normally tell me this at a glance”. Andrew Lo agrees; ultimately, fundamental and technical investors “should be able to learn from each other”.
Perfect For Beginners!
Free Trading Education!
Free Demo Account!
Good Choice For Experienced Traders!