Building a Data Science Workstation (2017)

For all the downsides of social media these days, the people I’ve met and the inside jokes bring me immense joy. From just this one tweet, several people messaged me aghast at 1) the idea of multi-gigabyte XML and that 2) I apparently have a computer for just this purpose!

Recently, I did build a workstation as a test bed for learning more about data science, specifically in the areas of Docker and GPU computing. Here’s what I built.

Machine Specs and Assembling from Parts

While there are several specialist workstation companies like Titan Computing that sell configurable workstations, assembling a computer from parts is pretty easy, and more importantly, can be significantly cheaper if you choose ‘consumer-grade’ parts instead of ‘server-grade’. Because I’m not planning on curing cancer or building Skynet with this computer, I opted not to get a dual-chip motherboard and used a single Intel i7 chip rather than going with Xeon workstation class chip(s). I also chose to use standard DDR4 RAM rather than ECC RAM. Here’s the full spec list:

PCPartPicker part list

Component Details
CPU Intel Core i7-5820K 3.3GHz 6-Core Processor
CPU Cooler Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler
Motherboard Asus X99-A II ATX LGA2011-3 Motherboard
Memory Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4-2400 Memory
Memory Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4-2400 Memory
Memory Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4-2400 Memory
Memory Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4-2400 Memory
Storage Samsung 850 EVO-Series 250GB 2.5” Solid State Drive
Storage Hitachi Deskstar 1TB 3.5” 7200RPM Internal Hard Drive
Video Card EVGA GeForce GTX 1060 6GB 6GB SC GAMING Video Card
Case Corsair SPEC-02 ATX Mid Tower Case
Power Supply Corsair CXM 750W 80+ Bronze Certified Semi-Modular ATX Power Supply
Optical Drive Asus DRW-24B1ST/BLK/B/AS DVD/CD Writer

Total: ~$2000

So for about $2000 and a few hours of time assembling, I have the rough equivalent of an r3.4xlarge instance on AWS ($1.33/hr on-demand at the time of writing). It would take about 1500 hours of usage to breakeven vs. using AWS, but cost isn’t really the point; the convenience of having the computer in my house and not having to do the startup/shutdown/EC2 images/S3/firewall/etc. dance is more than worth it to me so that I can focus on learning instead of operations.

Ubuntu 16.04LTS, CUDA, and Other Installs

While I did use Windows 10 to validate I put together my hardware correctly (and to mess with overlocking settings using the ASUS motherboard tools), I decided to use Ubuntu 16.04LTS as my base operating system. This allows for the most ‘server-like’ operations that I’m used to from a Linux environment. I enabled an internal static IP through my router, and for the most part, I either SSH into the machine from my MacBook Pro or use web UIs such as Jupyter Notebook (and again, remotely from my laptop).

I tried to install the NVIDIA drivers from source for the GTX1060 GPU, but eventually gave up and went the apt package manager route and everything works fine. Though I rarely sit at my desktop, I do have a 4K monitor hooked up to this computer which looks gorgeous with a video card of this caliber, and I have CUDA installed and working as well.

From there, I installed any number of tools from Python, R, Julia, MonetDB, Docker, Neo4j, Postgres, Spark, BlazingDB…any/all of which I hope to write about more in the near future.


RSiteCatalyst Version 1.4.10 Release Notes

Version 1.4.10 of RSiteCatalyst brings a handful of new Get* methods, QueueDatawarehouse and a couple of bugs fixes/low-level code improvements.

QueueDatawarehouse

The most useful user-facing change IMO is the addition of the QueueDatawarehouse method, which allows for submitting (and sometimes retrieving) Data Warehouse requests via R. This should be a huge timesaver for those of you using Data Warehouse as a substitute for the Adobe Analytics raw data feed (my employer alone pulls hundreds of Data Warehouse feeds per day).

In the coming days, I’ll write a blog post in more detail about how to use this method effectively to query Data Warehouse, but in the meantime, here’s a sample function call:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#API Credentials
SCAuth(Sys.getenv("USER", ""), Sys.getenv("SECRET", ""))
#FTP Credentials
FTP <- Sys.getenv("FTP", "")
FTPUSER <- Sys.getenv("FTPUSER", "")
FTPPW <- Sys.getenv("FTPPW", "")

#Write QueueWarehouse to FTP
report.id <- QueueDataWarehouse("report-suite",
                                "2016-11-01",
                                "2016-11-07",
                                c("visits", "pageviews"),
                                c("page"),
                                enqueueOnly=TRUE,
                                ftp = list(host = FTP,
                                           port = "21",
                                           directory = "/DWtest/",
                                           username = FTPUSER,
                                           password = FTPPW,
                                           filename = "myreport.csv")
)

ViewProcessingRules

ViewProcessingRules has also been added on an experimental basis, as this API method is not documented (as of the writing of this post), so Adobe may choose to modify or remove this in a future release. As the method name indicates, this new feature allows for the viewing of the processing rules for a list of report suites, including the behaviors that define the rules. There is currently no (public) method for setting processing rules via the API.

As processing rules are a super-user functionality, I would love it if someone in the community could verify that this method works for a large number of report suites/rules.

Get* method additions

Three Get* methods were added in this release: GetVirtualReportSuiteSettings, GetReportSuiteGroups, GetTimeStampEnabled, each corresponding to settings able to be viewed within the Adobe Analytics admin panel.

Bug fixes

  • GetRealTimeSettings: fixed bug where a passing a list of report suites lead to a parsing error

  • QueueSummary: Redefined method arguments to make date an optional parameter, allowing for more elegant use of date.to and date.from parameters.

Package-level improvements

At long last, I’ve added integration testing via the AppVeyor testing service. For the longest time, I’ve had a very nonchalant attitude towards Windows (since I don’t use it), but given that RSiteCatalyst is enterprise software and so many businesses use Windows, I figured it was time.

Luckily, there were none of the tests in the test suite throws any errors specifically due to Windows, so effectively this change is just defensive programming against that class of error in the future.

Community Contributions

As in the past several releases, there have been contributions from the community keeping RSiteCatalyst moving forward! Special thanks to Diego Villuendas Pellicero for writing the QueueDataWarehouse functionality and Johann de Boer for highlighting that QueueSummary could be improved.

I encourage all users of the software to continue reporting bugs via GitHub issues, and especially if you can provide a working code example. Even better, a fix via pull request will ensure that your bug will be addressed in a timely manner and for the benefit to others in the community.


WordPress to Jekyll: A 30x Speedup

About a month ago, I switched this blog from WordPress hosted on Bluehost to Jekyll on GitHub Pages. I suspected moving to a static website would be faster than generated HTML via PHP, and it is certainly cheaper (GitHub Pages is “free”). But it wasn’t until I needed a dataset for doing some dataset visualization development that I realize how much of an improvement it has been!

Packages, Packages, Packages

With the release of v0.5 of Julia, I’ve been working (less) on updating my packages and making new packages (more), because making new stuff is more fun than maintaining old stuff! One of the packages I’ve been building is for the ECharts visualization library (v3) from Baidu. While Julia doesn’t necessarily need another visualization library, visualization is something I’m interested in and learning is easier when you’re solving problems you like. And since the world doesn’t need another Iris example, I decided to share some real world website performance data :)

Line Chart

One of the first features I developed for ECharts.jl was X-Y charts, which I posit is the most common chart type in business. One thing that is great about the underlying ECharts JavaScript library is that interactivity is really easy to achieve:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
using ECharts, DataFrames

#Read in data
df = readtable("/assets/data/website_time_data.csv")

#Make data two different series that overlap, so endpoint touches
df[:pre] = [(x[1] <= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]
df[:post] = [(x[1] >= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]

#Graph code
l = line(df[:date], hcat(df[:pre], df[:post]))
l.ec_width = 800
seriesnames!(l, ["loadtime_ms", "post"])
colorscheme!(l, palette = ("acw", "FlatUI"))
yAxis!(l, name = "Load time in ms")
title!(l, text = "randyzwitch.com",
          subtext = "Switching from WordPress on Bluehost to Jekyll on GitHub (2016/09/06)")
toolbox!(l, chartTypes = ["bar", "line"])
slider!(l)

Even though I switched to Jekyll on WordPress on 9/6/2016, it appears that the page cache for Google Webmaster Tools didn’t really expire until 9/12/2016 or so. At the average case, the load time went from 1128ms to 38ms! Of course, this isn’t really a fair comparison, as presumably GitHub Pages runs on much better hardware than the cheap Bluehost hosting I have, and I didn’t reimplement most of the garbage I had on the WordPress version of the blog. But from a user-experience standpoint, good lord what an improvement!

Box Plots

Want to test out further functionality, here are some box plots of the load time variation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
using ECharts, DataFrames

#Read in data
df = readtable("/Users/randyzwitch/Desktop/website_load_time.csv")
df[:pre] = [(x[1] <= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]
df[:post] = [(x[1] >= "2016-09-12" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]

#Remove nulls
pre = [x for x in df[:pre] if x != nothing]
post = [x for x in df[:post] if x != nothing]

#Graph code
b = box([pre, post], names = ["WordPress", "Jekyll"])
b.ec_width = 800
colorscheme!(b, palette = ("acw", "VitaminC"))
yAxis!(b, name = "Load time in ms", nameGap = 50, min = 0)
title!(b, text = "randyzwitch.com",
           subtext = "Switching from WordPress on Bluehost to Jekyll on GitHub (2016/09/06)")
toolbox!(b)

Usually, a box plot comparison that is as smushed as the Jekyll plot vs the WordPress one would be a poor visualization, but in this case I think it actually works. The load time for the Jekyll version of this blog is so quick and so consistent that it barely registers as an outlier if it were WordPress! It’s crazy to think that the -1.5 * IQR time for WordPress is the mean/median/min load time of Jekyll.

Where To Go Next?

This blog post is really just an interesting finding from my experience moving to Jekyll on GitHub. As it stands now, ECharts.jl is stil in pre-METADATA mode. Right now, I assume that this would be a useful enough package to submit to METADATA some day, but I guess that depends on how much further I get smoothing the rough edges. If there are people who are interested in cleaning up this package further, I’d absolutely love to collaborate.


  • RSiteCatalyst Version 1.4.10 Release Notes
  • WordPress to Jekyll: A 30x Speedup
  • Bulk Downloading Adobe Analytics Data
  • Adobe Analytics Clickstream Data Feed: Calculations and Outlier Analysis
  • Adobe: Give Credit. You DID NOT Write RSiteCatalyst.
  • RSiteCatalyst Version 1.4.8 Release Notes
  • Adobe Analytics Clickstream Data Feed: Loading To Relational Database
  • Calling RSiteCatalyst From Python
  • RSiteCatalyst Version 1.4.7 (and 1.4.6.) Release Notes
  • RSiteCatalyst Version 1.4.5 Release Notes
  • Getting Started: Adobe Analytics Clickstream Data Feed
  • RSiteCatalyst Version 1.4.4 Release Notes
  • RSiteCatalyst Version 1.4.3 Release Notes
  • RSiteCatalyst Version 1.4.2 Release Notes
  • Destroy Your Data Using Excel With This One Weird Trick!
  • RSiteCatalyst Version 1.4.1 Release Notes
  • Visualizing Website Pathing With Sankey Charts
  • Visualizing Website Structure With Network Graphs
  • RSiteCatalyst Version 1.4 Release Notes
  • Maybe I Don't Really Know R After All
  • Building JSON in R: Three Methods
  • Real-time Reporting with the Adobe Analytics API
  • RSiteCatalyst Version 1.3 Release Notes
  • Adobe Analytics Implementation Documentation in 60 Seconds
  • RSiteCatalyst Version 1.2 Release Notes
  • Clustering Search Keywords Using K-Means Clustering
  • RSiteCatalyst Version 1.1 Release Notes
  • Anomaly Detection Using The Adobe Analytics API
  • (not provided): Using R and the Google Analytics API
  • My Top 20 Least Useful Omniture Reports
  • For Maximum User Understanding, Customize the SiteCatalyst Menu
  • Effect Of Modified Bounce Rate In Google Analytics
  • Adobe Discover 3: First Impressions
  • Using Omniture SiteCatalyst Target Report To Calculate YOY growth
  • Google Analytics Individual Qualification (IQ) - Passed!
  • Google Analytics SEO reports: Not Ready For Primetime?
  • An Afternoon With Edward Tufte
  • Google Analytics Custom Variables: A Page-Level Example
  • Xchange 2011: Think Tank and Harbor Cruise
  • Google Analytics for WordPress: Two Methods
  • WordPress Stats or Google Analytics? Yes!
  • Building a Data Science Workstation (2017)
  • JuliaCon 2015: Everyday Analytics and Visualization (video)
  • Vega.jl, Rebooted
  • Sessionizing Log Data Using data.table [Follow-up #2]
  • Sessionizing Log Data Using dplyr [Follow-up]
  • Sessionizing Log Data Using SQL
  • Review: Data Science at the Command Line
  • Introducing Twitter.jl
  • Code Refactoring Using Metaprogramming
  • Evaluating BreakoutDetection
  • Creating A Stacked Bar Chart in Seaborn
  • Visualizing Analytics Languages With VennEuler.jl
  • String Interpolation for Fun and Profit
  • Using Julia As A "Glue" Language
  • Five Hard-Won Lessons Using Hive
  • Using SQL Workbench with Apache Hive
  • Getting Started With Hadoop, Final: Analysis Using Hive & Pig
  • Quickly Create Dummy Variables in a Data Frame
  • Using Amazon EC2 with IPython Notebook
  • Adding Line Numbers in IPython/Jupyter Notebooks
  • Fun With Just-In-Time Compiling: Julia, Python, R and pqR
  • Getting Started Using Hadoop, Part 4: Creating Tables With Hive
  • Tabular Data I/O in Julia
  • Hadoop Streaming with Amazon Elastic MapReduce, Python and mrjob
  • A Beginner's Look at Julia
  • Getting Started Using Hadoop, Part 3: Loading Data
  • Innovation Will Never Be At The Push Of A Button
  • Getting Started Using Hadoop, Part 2: Building a Cluster
  • Getting Started Using Hadoop, Part 1: Intro
  • Instructions for Installing & Using R on Amazon EC2
  • Video: SQL Queries in R using sqldf
  • Video: Overlay Histogram in R (Normal, Density, Another Series)
  • Video: R, RStudio, Rcmdr & rattle
  • Getting Started Using R, Part 2: Rcmdr
  • Getting Started Using R, Part 1: RStudio
  • Learning R Has Really Made Me Appreciate SAS