Posts

Webscraping in R - IMDb ETL Showcase

Image
  Web scraping in R is an ETL pipeline that perform web data mining by reading HTML tags and converting them  to the structured format which can easily be visualized using tidyverse . Let's  scrape movies from IMDb into a data frame in R by invoking the rvest library and then visualize the data frame using ggplot2 and qplot functions: Importing the key R libraries library(rvest) #scraping library(dplyr) #piping library('ggplot2') #plotting Specifying the URL for desired website to be scraped url <- 'http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature' Reading the HTML code from the website webpage <- read_html(url) Using CSS selectors to scrape the rankings section rank_data_html <- html_nodes(webpage,'.text-primary') Converting the ranking data to text rank_data <- html_text(rank_data_html) Let's have a look at the rankings head(rank_data) [1] "1." "2." "3." "4.&q

Short-term Stock Market Price Prediction using Deep Learning Models

Image
  This blog is about short-term stock market price trend prediction using a comprehensive deep learning LSTM model. Results show that t he model achieves overall high accuracy for stock market trend prediction .  The following end-to-end sequence  provides the detailed Python/Jupyter workflow from data processing to prediction, including the data exploration: 1. Data Preparation Phase #import libraries import pandas as pd import numpy as np # To remove the scientific notation from numpy arrays np.set_printoptions(suppress=True)   # install the nsepy library to get stock prices !pip install nsepy   ############################################ # Getting Stock data using nsepy library from nsepy import get_history from datetime import datetime   startDate=datetime(2021, 1,1) endDate=datetime(2022, 10, 3)   # Fetching the data StockData=get_history(symbol='INFY', start=startDate, end=endDate) print(StockData.shape) StockData.head() # Creating a column as date StockData['TradeDa

Ml/AI Regression for Stock Prediction - AAPL Use Case

Image
  The following is a set of steps intended for ML/AI regression to predict stock prices. The objective is to simulate available historical stock prices of $AAPL using the SciKit Learn library. 1. Install Yahoo finance library !pip install yfinance 2. Let's call  all dependencies that we will use for this exercise  import pandas as pd import numpy as np import math  import seaborn as sns  import matplotlib.pyplot as plt from sklearn import metrics from sklearn.model_selection import train_test_split import yfinance as yf  # We will use this library to upload latest data from Yahoo API %matplotlib inline plt.style.use('fivethirtyeight') 3. Define the ticker you will use nio = yf.Ticker('AAPL') #Display stock information, it will give you a summary description of the ticker nio.info {'zip': '95014', 'sector': 'Technology', 'fullTimeEmployees': 100000, 'longBusinessSummary': 'Apple Inc. designs, manufactures, and