In this post, we’ll walk through the basic concepts , properties, and practical values of Normal distribution. If you want to discuss anything or find an error, please email me at Lvqingchuan@gmail.com :) Warm-up Before getting started, I want you to take a close look at the above diagram. This depicts…

In this post, we’ll walk through basic concepts behind Random Forest, discuss practical problems of implementation, such as highly correlated features, feature sparsity, imbalanced classes. Then, compare the concepts and performance of Random Forest to Boosting Trees and Decision Tree! If you want to discuss anything or find an error…

This post is part of efforts on improving my technical projects. Each post describes how I identified suspicious spots and updated them. You can find Part I: Regressors (Price) and Part II: Time Series (Sales) here. …

This post is part of efforts on improving my technical projects. Each post describes how I identified suspicious spots and updated them. You can find Part I: Regressors and Part III: NLP here. …

In this post, I discuss how I edited feature engineering and modeling part. This post is the first part of “Redoing Projects” series. You can find Part II: Time Series (Sales) and Part III: Tweet Sentiment Extraction (NLP) here. Brief Background This project aims at predicting house prices in Ames, Iowa with…

In this post, I’ll show you necessary assumptions for linear regression coefficient estimates to be unbiased, and discuss other “nice to have” properties. There are many versions of linear regression assumptions on the internet. Hopefully, this post will make it clear. “Must have” Assumption 1. …

In this post, we’ll walk through basic techniques of data manipulation and simulation with Python. Data manipulation A. Group aggregation Given a data frame, say we want to summarize customer orders by different genders, we can use a simple groupby and agg function: values_gender = csv_file\ .groupby(['gender']) \ .agg(avg_order_values=('value','mean'),\ …

In this post, we’ll learn what is power, significant level, type I/II errors and how they relate to each other. Power Once we setup the null hypothesis and alternative hypothesis, we will collect data and compute test statistics. …