Here are the notes Prof. Widom used for the presentation on data analysis techniques: DataAnalysisTechniques.pdf.


Here are the data sets and other graphics used in class:

Basic database operations

• Temperatures table: Temps.csv
• Filtering: VeryCold.csv
• Sorting: SouthNorth.csv
• Aggregating: AvgTemp.csv, AvgByState.csv
• Joining: Regions.csv, JoinedTables.csv
• Composing operations: GrandFinale.csv

Regression

• Temp vs latitude: TempVsLat.pdf, TempVsLatRegression.png
• Temp vs longitude: TempVsLong.pdf, TempVsLongRegression.png
• Underfitting, overfitting, limitations: UnderOverFitting.png, AnscombesQuartet.svg

Classification

• K nearest neighbors (KNN)
  • Temperature example: TempsCat.csv, LatLongScatter.png
  • Use for regression: LatLongScatterTemps.png
• Decision tree classifier
  • Temperature example - predict category from feature: CatNoTemps.csv, CatNoTempsSorted.csv
• Naive Bayes: probabilistic
  • CategoryProbabilities.csv
  • ConditionalProbabilities.csv
• Underfitting and overfitting in classification
  • President Overfitting

Clustering

• Temperature example - cluster cities into six groups based on latitude/longitude: Points.png, Clusters.png, ClusterMeans.png

Here are some additional readings covering many of the techniques:

• Data Mining
  • Association rule learning
• Regression
  • Introduction to Linear Regression
  • Anscombe’s quartet
• Classification
  • k-Nearest neighbors algorithm
  • KNN - Classification
  • Decision tree learning
  • Naive Bayes classifier
  • Naive Bayes
• Clustering
  • Cluster analysis
  • k-Means clustering 1
  • k-Means clustering 2