Stock Markets Analytics Zoomcamp 2024

Homework 3: Modeling for Financial Time Series Statistics

Distribution of scores and reported study time for this homework.

Submissions

64

Median total score

6

Average total score

6

Score distribution

All values are points.

Questions score

Min
1
Median
6.0
Max
7
Q1
4.0
Avg
5.4
Q3
7.0

Learning in public score

Min
-
Median
0.0
Max
7
Q1
0.0
Avg
0.9
Q3
1.0

Total score

Min
1
Median
6.0
Max
14
Q1
4.0
Avg
6.3
Q3
7.0

Time distribution

All values are hours reported by students.

Lectures

Min
0.5
Median
3.0
Max
30.0
Q1
2.0
Avg
4.6
Q3
5.3

Homework

Min
2.0
Median
6.5
Max
20.0
Q1
4.0
Avg
7.6
Q3
11.5

Question breakdown

Correctness and answer distribution per question.

1. [Dummies on Month and Week-of-Month] Top correlation of a new dummy with a boolean prediction variable?

53 / 64 correct (82.8%)

1 0.025 6 (9.4%)
2 0.035 53 (82.8%)
3 0.045 0 (0.0%)
4 0.055 4 (6.2%)

2. [Define new 'hand' rules] What's the precision score for the best new rule?

55 / 64 correct (85.9%)

1 0.575 1 (1.6%)
2 0.591 1 (1.6%)
3 0.612 4 (6.2%)
4 0.555 55 (85.9%)

3. [Unique correct predictions for pred5_clf_10] Wha'ts the total number of records of 'unique correctness' on TEST?

33 / 64 correct (51.6%)

1 1 33 (51.6%)
2 5 6 (9.4%)
3 10 5 (7.8%)
4 15 4 (6.2%)
5 20 6 (9.4%)

4. [Hyperparams tuning] What's the best max_depth for a Decision Tree Classifier?

44 / 64 correct (68.8%)

1 13 3 (4.7%)
2 16 1 (1.6%)
3 15 44 (68.8%)
4 18 3 (4.7%)
5 20 3 (4.7%)

5. [EXPLORATORY] What data features are missing?

64 / 64 correct (100.0%)

Answer Count
Perhaps other economic macroindicators. E.g. Unemployment rate 1
keine Ahnung 1
Sazonality: It should consider the quarterly and yearly sazonality as it's the release period of the company balance. 1
We have lots of data features added. Additional data features can be added based on analysis need. 1
It seems like commodities (i.e., gold, oil) are important features in the model, so I would suggest including some other commodities in the model, such as silver, platinum, natural gas, and corn. The 30-day and 365-day growth indicators seem to have some explanatory success, so I would use these time ranges for the suggested commodity features. 1
Likely the missing features are SMAs and EMAs of the important growth indicators such as gold, oil, fastd (which are indicated by decision tree classifier). Correlation analysis showed that certain months like September, March and October are highly correlated with the predictive variable and also the week within the month makes a difference. This suggests that short term trends in the growth indicators could be important indicators. 1
see hw3-notebook.ipynb 1
There are several indicators that could enhance the predictive power of a dataset. These indicators can be grouped into different categories: technical indicators, fundamental indicators, and macroeconomic indicators. More information provided on the Colab. 1
Depending on the investment, it would be nice to have a (daily, 4-hour, 15-min) or (1-hour, 15-min, 5-min) trend indicator. Then if all time frames indicate an up or downtrend, then a correspsonding buy or sell signal could be indicated and a jump in point could be found to jump into the trade. 1
VIX 1
To determine what data might be missing and suggest new indicators to include in the dataset, let's first consider what insights you might have gained from the correlation analysis and the Decision Trees. Typically, these methods help identify which variables have the strongest relationships with the target variable (e.g., stock performance). Based on common influential factors in stock performance, here are some suggestions: New Indicators to Include Macroeconomic Indicators Including these can provide context about the economic environment which influences stock performance. GDP Growth Rate: Measures the economic growth of a country, impacting corporate earnings. Inflation Rate: Affects purchasing power and cost of borrowing. Interest Rates: Central bank rates impact consumer spending and business investments. Unemployment Rate: Reflects economic health and consumer confidence. Market Sentiment Indicators Market sentiment can significantly impact stock prices. Social Media Sentiment: Analyze sentiment from platforms like Twitter and Reddit. News Sentiment: Analyze sentiment from financial news articles. Industry-Specific Indicators These can provide insights specific to different sectors. Commodity Prices: For sectors dependent on raw materials (e.g., oil for energy companies). Regulatory Changes: Information about new regulations affecting specific industries. Corporate Governance Indicators These can give insights into company management quality and governance. Board Composition: Diversity and independence of the board. Executive Compensation: Alignment with shareholder interests. ESG Scores: Environmental, Social, and Governance scores. Technical Indicators Useful for understanding stock price movements and trends. Relative Strength Index (RSI): Identifies overbought or oversold conditions. Moving Averages (SMA and EMA): To identify trends. Foreign Exchange Rates For companies with significant international operations, exchange rates impact earnings. Exchange Rates: Include USD/EUR, USD/INR, EUR/INR. Political Stability and Policy Changes Political events can influence investor confidence. Political Stability Index: Measures the likelihood of political instability. Policy Changes: Major policy shifts such as tax reforms or trade agreements. Potential Data Sources World Bank and IMF: For macroeconomic indicators. Commodity Markets: Platforms like Bloomberg and Reuters for commodity prices. Social Media Platforms: APIs from Twitter, Reddit for sentiment analysis. News Outlets: APIs for extracting news sentiment. Corporate Filings: SEC, ESMA, and Indian MCA for governance data. Forex Platforms: For real-time exchange rates (e.g., XE, OANDA). Why Include These Indicators? Contextual Understanding: Macroeconomic indicators provide context about the broader economic environment impacting all stocks. Sentiment Insight: Sentiment analysis helps gauge public and investor sentiment which can drive short-term market movements. Industry Specificity: Industry-specific data help tailor the analysis to particular sectors, recognizing that different sectors are influenced by different factors. Governance Quality: Corporate governance indicators can help assess the quality and sustainability of a company’s management practices. Technical Analysis: Technical indicators help in understanding price trends and making trading decisions. Currency Impact: For multinational companies, currency fluctuations can have significant impacts on their financials. Political Context: Political stability and policy changes can have major effects on market confidence and economic conditions. Including these additional indicators will provide a more holistic view and improve the predictive power of your models by capturing a wider array of factors that influence stock performance. 1
Null 1
The technical indicators are using fixed time periods. By varying the time periods, a model could assess whether price action is more directly correlated with various indicators and also whether certain indicators and time periods have higher accuracy and precision when they follow various economic and fundamental indicators. 1
some other features we could look at: US unemployment rate EU, Asia growth rate number of IPOS planned in quarter paid dividends up to today in comparison to last year in different markets 1
some new indicators such as revenue and earnings growth, sentiment analysis, and technological trends. Thanks to artificial intelligence can act as a catalyst for some semiconductor industries, especially in the Graphics Processing Unit (GPU) which vital role in processing in AI 1
Proposed Indicators: Global Economic Indicators: Consumer Confidence Index: Influences market behavior through consumer spending. Source: The Conference Board (Link) (USA), Eurostat (Link) (EU), RBI (Link) (India). Industrial Production Index: Key indicator of economic activity in manufacturing, mining, and utilities. Source: Federal Reserve (Link) (USA), Eurostat (Link) (EU), MOSPI (Link) (India). Geopolitical and Social Factors: Political Risk Index: Quantifies the impact of political events on financial markets. Source: World Bank (Link), PRS Group (Link). Unemployment Rates: Indicates economic health; high rates signal economic issues. Source: Bureau of Labor Statistics (Link) (USA), Eurostat (Link) (EU), MOSPI (Link) (India). Sectoral Indicators: Retail Sales Index: Reflects consumer spending and can predict economic changes. Source: U.S. Census Bureau (Link) (USA), Eurostat (Link) (EU), MOSPI (Link) (India). Key Commodity Prices: Includes copper prices, indicating global industrial activity. Source: London Metal Exchange (Link) (LME). Market Sentiment Indicators: Fear & Greed Index: Measures market sentiment and predicts stock price movements. Source: CNN Business (Link). Climatic and Environmental Data: Climate Risk Index: Assesses the impact of climate change and natural disasters on markets. Source: Germanwatch (Link), World Bank (Link). 1
Ratio of positive and negative news (weekly, daily), Stocks price vs Revenue, Quarterly Financial report vs Predicted Financial report values 1
Technical indicators like bolinger band, VWAP etc. Further, we can try on all possible combinations of features in our current approach to select top few with best precision (i.e. bucket continuous features, combine (multiply) with other categorical features to get new feature.) 1
maybe something related to holiday/business days or even days after financial crisis. 1
Inflation Rate, Unemployment Rate, Interest Rate, Industry-Specific Indexes, Commodity Prices, Relative Strength Index (RSI), Bollinger Bands, Social Media Sentiment, News Sentiment, Foreign Exchange Rates 1
will explore once I have time 1
Fundamental Earnings Per Share (EPS) Measures a company's profitability. Higher EPS indicates better profitability. Yahoo Finance, Morningstar, Reuters Fundamental Price-to-Earnings Ratio (P/E) Evaluates a company's current share price relative to its per-share earnings. Yahoo Finance, Morningstar, Reuters Fundamental Price-to-Book Ratio (P/B) Compares a firm's market value to its book value. Yahoo Finance, Morningstar, Reuters Fundamental Dividend Yield Indicates how much a company pays out in dividends each year relative to its share price. Yahoo Finance, Morningstar, Reuters Fundamental Debt-to-Equity Ratio (D/E) Measures a company's financial leverage. Higher D/E ratios can indicate higher risk. Yahoo Finance, Morningstar, Reuters Technical Relative Strength Index (RSI) A momentum oscillator that measures the speed and change of price movements. Yahoo Finance, TradingView, Alpha Vantage API Technical Moving Averages (MA) Used to smooth out price data and identify trends (e.g., SMA, EMA). Yahoo Finance, TradingView, Alpha Vantage API Technical Moving Average Convergence Divergence (MACD) A trend-following momentum indicator that shows the relationship between two moving averages. Yahoo Finance, TradingView, Alpha Vantage API Economic Interest Rates Changes in interest rates can impact stock prices, particularly in interest-sensitive sectors. FRED, World Bank, OECD Statistics Economic Inflation Rate Higher inflation rates can erode purchasing power and affect consumer spending. FRED, World Bank, OECD Statistics Economic GDP Growth Rate Indicates the overall economic health and growth of a country. FRED, World Bank, OECD Statistics Economic Unemployment Rate A higher unemployment rate can indicate economic distress, which may negatively impact stock performance. FRED, World Bank, OECD Statistics 1
See the Question 5 part in the Homework URL. 1
Suggested New Data Features: Technical Indicators: - Moving Averages (50-day, 200-day): To identify trends and potential buy/sell signals. - Relative Strength Index (RSI): To measure the speed and change of price movements, indicating overbought or oversold conditions. - MACD (Moving Average Convergence Divergence): To identify changes in the strength, direction, momentum, and duration of a trend. Market Indicators: - Trading Volume: To provide insight into the liquidity and activity level of the stocks. - Market Capitalization: To categorize companies into large-cap, mid-cap, and small-cap, which can impact performance and volatility. Corporate Actions: Stock Splits/Reverse Splits: Historical data on stock splits can affect stock price and investor perception. Mergers and Acquisitions: Information on past mergers and acquisitions to understand growth strategies and market impact. Economic Indicators: Unemployment Rates: Country-specific unemployment data that can impact consumer spending and business performance. Consumer Confidence Index: Measures consumer sentiment, which can affect spending and investment behavior. 1
Additional indicators like economic metrics (GDP growth, unemployment rate), market sentiment indicators (VIX, put/call ratio), sector-specific data (oil prices, commodity prices), global indicators (foreign exchange rates, interest rates), and corporate metrics (earnings reports, dividend yields) are missing; they provide insights into macroeconomic conditions, investor sentiment, sector trends, global influences, and company performance. 1
Incorporating additional indicators enhances the dataset by providing a more comprehensive view of the economic, market, and corporate environment. VIX and CCI offer insights into market sentiment and consumer behavior, which are crucial for anticipating market movements. Inflation rates, interest rate spreads, and crude oil prices capture essential macroeconomic factors influencing costs and profitability. Foreign exchange rates affect international trade and the financial performance of multinational corporations. Corporate earnings growth provides a direct measure of company health, while employment rates reflect the overall economic activity. Additionally, technology adoption and ESG scores highlight innovation and sustainability trends, which are increasingly important for long-term growth. By integrating these diverse indicators, we can improve predictive models and develop more robust investment strategies. 1
To enhance the analysis of the largest Indian, EU, and US stocks, including macroeconomic indicators such as GDP growth rate, inflation rate, and unemployment rate is essential. These indicators significantly influence stock prices by reflecting economic health and consumer purchasing power. Additionally, industry-specific indicators like commodity prices and industry growth rates offer insights into sector-specific performance, while ESG scores can attract socially conscious investors, potentially boosting stock prices. 1
A number of other indicators are missing that shed light on macroeconomic conditions, investor sentiment, and company performance. Examples of these indicators include economic metrics (such as GDP growth and the unemployment rate), and corporate metrics (such as dividend yields and earnings reports). 1
The choice of additional indicators should be guided by the specific research questions and objectives of the analysis. It's essential to prioritize data sources that are reliable, relevant, and have a meaningful impact on stock price movements. Additionally, combining multiple types of data can provide a more comprehensive understanding of the factors driving stock market behavior Note: Not all of below indicators can be directly included in our current dataset new_df as is. Market Sentiment Indicators: Including data on market sentiment, such as sentiment scores from news articles, social media platforms, or analyst reports, could provide valuable insights into market dynamics. Sentiment analysis can capture the mood and perception of investors, which can influence stock prices. Sector-specific Metrics: Since the dataset covers stocks from various sectors, including sector-specific metrics such as industry growth rates, sector-specific news sentiment, or regulatory changes relevant to each sector can provide deeper insights into the performance of individual stocks. Technical Indicators: Adding technical indicators such as moving averages, relative strength index (RSI), and moving average convergence divergence (MACD) can help capture short-term price trends and momentum in the stock market. These indicators are widely used by technical analysts for making trading decisions. Geopolitical Events: Considering geopolitical events and their potential impact on stock markets could be valuable. Events like elections, trade tensions, geopolitical conflicts, and policy changes can significantly influence investor sentiment and market volatility. Fundamental Analysis Data: Incorporating fundamental analysis data such as earnings per share (EPS), price-to-earnings (P/E) ratio, and book value can provide insights into the underlying financial health and valuation of companies. Alternative Data Sources: Exploring alternative data sources such as satellite imagery analysis, web scraping of e-commerce platforms for sales data, or credit card transaction data can offer unique insights into company performance and consumer behavior. 1
I would like to know more about the companies financial results to assess for example debt ratios, cash flow indicators, ebitda, among others so I can perform a more value-based analysis as well. 1
--- 1
Based on the fact that `growth_wti_oil_365d` and `growth_wti_oil_30d` are among top important features, as well as `growth_gold_365d` and `growth_gold_7d`, I would create new feature type for constant growth (that is without any declines). 1
Insider Trading Activities Description: Include data on insider trading activities from SEC filings and stock exchange disclosures. Importance: Insider trades can indicate insiders' confidence in their company's future performance, influencing investor decisions." Sector-Specific Indices Description: Compare individual stocks with their respective sector indices, such as the technology or healthcare sector index. Importance: Understanding how a stock performs relative to its sector can provide insights into its competitive positioning and sectoral trends. Environmental, Social, and Governance (ESG) Scores Description: Incorporate ESG scores from rating agencies like MSCI and Sustainalytics to evaluate companies' sustainability and ethical practices. Importance: ESG factors are increasingly important to investors and can impact stock performance based on a company's commitment to sustainability and good governance. 1
Using statistical model to extract seasonal trend as features 1
To enhance the dataset with additional sentiment indicators, I suggest incorporating market volatility, specifically the VIX (Volatility Index). The VIX measures market expectations of near-term volatility and is often referred to as the "fear index." Including this indicator can provide valuable insights into market sentiment and risk perception. 1
According of what I read, one important indicator is interest rates, and inflation rates. 1
market capitalization 1
New and social media sentiment, macro forecast, technical indicatos 1

Calculated: 13 October 2024, 16:30