Real Estate Supervised ML/AI Linear Regression Revisited - USA House Price Prediction

 

ETL Workflow

Linear regression is an algorithm of supervised Machine Learning (ML) in which the predicted output is continuous with having a constant slope [1]. Consider a company of real estate with datasets containing the property prices of a specific region. The price of a property is based on essential factors like bedrooms, areas, and parking.  Majorly, a real estate company requires [1]:

  • Define the set of variables (model features like areas, number of rooms and bathroom, etc.) that affects the price of a house;
  • Creating a linear model quantitatively related to the house price with model variables;
  • Examine the accuracy of an output model, i.e. how well the model variables can predict the prices of a house for training, test and validation data.
So real estate experts assume that the trained, tested and deployed ML model would be capable of learning and predicting how far people would go in the bidding in order to buy a house, based on selected features. That’s the ML housing price prediction algorithm in a nutshell.

Step 1: Exploratory Data Analysis (EDA)

The public domain dataset USA_Housing.csv [2] contains 7 columns and 10000 rows with CSV extension. The data contains the following columns:


  • 'Avg. Area Income' - Scaled Average Income of the householder of the city where house is located,
  • 'Avg. Area House Age' - Scaled Average age of houses in the same city.
  • 'Avg. Area Number of Rooms' - Scaled Average number of rooms for houses in same city.
  • 'Avg. Area Number of Bedrooms' - Scaled Average number of bedrooms for houses in the same city.
  • 'Area Population' - The scaled total population of the city.
  • 'Price' - Scaled price that the hose sold at.
  • 'Address' - Unscaled address of that house.
All attributes are numeric except for the "Address" field. Its type is an object, so it can contain any type of Python object. A quick way to get a feel for what kind of data you’re dealing with is to plot a histogram and a box/whisker plot for each numerical attribute, as shown below.















Figure 1: Histograms and box/whisker plots of raw input data [2].


The actual steps of EDA and linear regression [3] are as follows:








We see that Avg. area Income, Area Population and Avg. Area House Age have the largest correlation with house prices.






The sensitivity coefficient means the following:
1. Holding all the other features fixed, a 1 unit increase in Avg. Area Income is associated with an increase of $21.66.
2. Holding all the other features fixed, a 1 unit increase in Avg. Area House Age is associated with an increase of
$164990.05
3. Holding all the other features fixed, a 1 unit increase in Avg. Area Number of Rooms is associated with an increase of
$120784.23
4. Holding all the other features fixed, a 1 unit increase in Avg. Area Number of Bedrooms is associated with an increase
of $1542.52
5. Holding all the other features fixed, a 1 unit increase in Area Population of Bedrooms is associated with an increase
of $15.15
Thus, we can neglect features 1, 4 and 5 as compared to features 2 and 3.

In the above scatter X-plot, we see test versus predictions are of a line form, which means our model has done good predictions.


In the above histogram plot we see data is in bell shape(Normally Distributed), which means our model has done good predictions.

References






















Comments

Post a Comment

Popular posts from this blog

Short-term Stock Market Price Prediction using Deep Learning Models

Image
  This blog is about short-term stock market price trend prediction using a comprehensive deep learning LSTM model. Results show that t he model achieves overall high accuracy for stock market trend prediction .  The following end-to-end sequence  provides the detailed Python/Jupyter workflow from data processing to prediction, including the data exploration: 1. Data Preparation Phase #import libraries import pandas as pd import numpy as np # To remove the scientific notation from numpy arrays np.set_printoptions(suppress=True)   # install the nsepy library to get stock prices !pip install nsepy   ############################################ # Getting Stock data using nsepy library from nsepy import get_history from datetime import datetime   startDate=datetime(2021, 1,1) endDate=datetime(2022, 10, 3)   # Fetching the data StockData=get_history(symbol='INFY', start=startDate, end=endDate) print(StockData.shape) StockData.head() # Creating a column as date StockData['TradeDa
Read more

Webscraping in R - IMDb ETL Showcase

Image
  Web scraping in R is an ETL pipeline that perform web data mining by reading HTML tags and converting them  to the structured format which can easily be visualized using tidyverse . Let's  scrape movies from IMDb into a data frame in R by invoking the rvest library and then visualize the data frame using ggplot2 and qplot functions: Importing the key R libraries library(rvest) #scraping library(dplyr) #piping library('ggplot2') #plotting Specifying the URL for desired website to be scraped url <- 'http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature' Reading the HTML code from the website webpage <- read_html(url) Using CSS selectors to scrape the rankings section rank_data_html <- html_nodes(webpage,'.text-primary') Converting the ranking data to text rank_data <- html_text(rank_data_html) Let's have a look at the rankings head(rank_data) [1] "1." "2." "3." "4.&q
Read more

Supervised ML/AI Stock Prediction using Keras LSTM Models

Image
  (the image was created using Visme [1]). Introduction Stock markets are analyzed either technically or fundamentally [2]. Fundamental analysis studies supply and demand relationships that define the stock price at any given time.  Technical analysis uses specialized methods of predicting prices by analyzing past price patterns and levels.  T here are many techniques used to examine stock price lines and patterns [2]: bar or high/low/close charts moving averages trend lines channels cycles resistance and support planes corrections double tops and bottoms head and shoulders formation trading volume open interest.  Theere are numerous limitations of these techniques: moving averages responds to general trends only is not highly precise short-term moving averages can give false indications, especially in times of volatile prices Trend lines work best with sustained trends positioning of trend lines is subjective and takes practice trends must be established before they become recognizabl
Read more