Despite my preference for SAS over R, there are some add-ons to “basic” R that I’ve found that have made my learning process way easier. While I’m still in my infancy in learning R, I feel like once I found these additional tools, my ability to use R to get work done improved significantly.
In this first post of three, I’ll discuss RStudio, a more friendly access point to the default installation of R. My second post will discuss Rcmdr, a GUI developed for students taking a basic college-level course in Statistics. The third post will cover rattle, a GUI specifically designed for data mining (as opposed to more general statistics like Rcmdr).
R Studio is an IDE that dramatically improves the R experience
RStudio is an open-source Integrated Development Environment (IDE) that provides a more consistent user experience to R. There are many great features of RStudio over “basic” R, including:
- Consistent windowing between sessions (customizable by the user)
- Point-and-click exploration of data frames and other data objects
- Importing data files through dialog box functionality
- Customizable code syntax highlighting, auto-complete, and Help menu access from the code editor
- Ability to see all installed packages, turn on packages using a checkbox, and download libraries (and their dependencies) without having to write any code
- Version Control using GitHub
While RStudio doesn’t provide a GUI that will help you run a regression model or build a graph, it provides a more “friendly” environment to work in as compared to the command-line interface of a default installation of R. I find that by having elements like the currently active data objects and available/active packages with links to the Help files “exposed” at all times, RStudio reminds me of where my analysis has been and gives me a quick way to think about “What Else?” to pursue if I hit a roadblock.
Installation of RStudio
RStudio installs like any other program for Windows or Mac OSX. As far as I can tell, there are no advantages to using RStudio in either environment, both the Windows and OSX versions seem to work equally well. The most important consideration is that RStudio is just an “add-on” so-to-speak, it does not include R itself. So be sure to go to one of the Comprehensive R Archive Network (CRAN) sites to download R first.
EDIT, 9/9/2016: Four years later, this blog post is a comical look back in time. It’s hard to believe that I could think this way! Having used R (and Python, Julia), I will never return back to the constraints of using SAS. The inflexible nature of everything having to be a Dataset in SAS vs. the infinite flexibility of data structures in programming-oriented languages makes it no contest.
But I’ll leave this here to remind myself how today’s frustration leads to tomorrow’s breakthroughs.
For the past 18 months, it seems like all I’ve heard about in the digital marketing industry is “big data”, and with that, mentions of using Hadoop and R to solve these sorts of problems. Why are these tools the most often mentioned? Because they are open source, i.e. free of charge!
But as I’ve tried to learn R, I keep asking myself…are all of my colleagues out of their minds? Or, am I just beyond learning something new? As of right now, R is just one big hack on top of a hack to me, and the software is only “free” if you don’t consider lost productivity.
Need new functionality, just download another R package!
One of the biggest “pros” I see thrown around for R relative to a tool like SAS is that when new statistical techniques are invented, someone will code it in R immediately. A company like SAS make take 5 years to implement the feature, or it may not get implemented at all. That’s all fine and good, but the problem I’ve found is that there are 10 ways to do something in R, and I spend more time downloading packages (along with other packages that are dependencies) than I do learning A SINGLE WAY to do something correctly.
For example, take trying to get summary statistics by group. In SAS, you use a Proc Summary statement, with either a BY group statement or a CLASS statement. It’s fairly simple and it works.
proc summary data= hs0; var _numeric_; class prgtype; output out=results mean= /autolabel autoname inherit; run;
In R, I ran the following code, which should be roughly equivalent:
by(hs0, hs0$prgtype, mean)
Very simple, fewer lines…and technically wrong, throwing a 6 unhelpful errors for a single line of code. Because it was decided that “mean” as a function would be deprecated in R. WHY??? It’s so simple, why modify the language like that?
According to the error message, I’m supposed to use colMeans instead…but once you get to how, you’re on your own, the Help documentation is garbage. Some combination of “by” and “colMeans” might work, but I don’t have an example to follow.
Google sent me to the Quick-R website, and I found a “descriptive statistics” article with by group processing…with the recommendation of using the “psych” package or the “doBy” package. But CRAN won’t let me download all of the dependencies, so again, stuck trying to do the simplest thing in statistics.
Let’s be fast and run everything in RAM!
My next favorite hassle in R is that you are expected to continuously monitor how many data elements you have active in a workspace. R runs completely in RAM (as opposed to SAS which runs a combination of RAM for processing and hard disks for storage), so if you want to do something really “big”, you will quickly choke your computer. I tried to work with a single day of Omniture data from the raw data feed, and my MacBook Pro with 6GB of memory was shot. I believe the file was 700,000 rows by 300 columns, but I could be mis-remembering. That’s not even enough data to think about performance-tuning a program in SAS, any slop code will run quickly.
How does one solve these memory errors in R? Port to Amazon cloud seems to be the most commonly given suggestion. But that’s more setup time, getting an R instance over to Amazon, your data over to Amazon..and now you are renting hardware.
R is great for data visualization!
From what I’ve seen from the demo(graphics) tutorial, R does have some pretty impressive visualization capabilities. Contour maps, histograms, boxplots…there seems to be a lot of capability here beyond the realm of a tool like Excel (which, besides not being free, isn’t really for visualization). SAS has some graphics capabilities, but they are a bit hard to master.
But for all of the hassle to get your data formatted properly, downloading endless packages, avoiding memory errors, you could just pay for Tableau and get working. Then, once you have your visualizations done in Tableau, if you are using Tableau server you can share interactive dashboards with others. As far as I know, R graphics are static image exports, so you’re stuck with “flat” presentations.
Maybe, it’s just me
For R diehards, the above verbiage probably just sounds like whining from someone who is too new to appreciate the greatness of R or too stuck in the “old SAS way”. That’s certainly possible. But from my first several weeks of trying to use R, the level of frustration is way beyond anything I experienced when I was learning SAS.
Luckily, I don’t currently have any consulting projects that require R or SAS at the moment, so I can continue to try and learn why everyone thinks R is so great. But from where I sit right now, the licensing fee from SAS doesn’t seem so bad when it allows me to get to doing productive work instead of building my own statistics software piece-by-piece.
Just because data CAN be captured doesn't mean it SHOULD be!
In a prior post about customizing the SiteCatalyst menu interface, I discussed how simple changes such as hiding empty Omniture variables/reports and re-organizing the menu structure will help improve understanding within your organization. In the spirit of even further interface optimization, here are 20 reports within Omniture that I feel that can be hidden due to their lack of business-actionable information.
Here are my Top 20, in no particular order:
- Mobile: Color Depth
- Mobile: Information Services
- Mobile: Decoration Mail Support
- Mobile: PTT
- Mobile: Device Number Transmit
- Mobile: Browser URL Length
- Mobile: DRM
- Mobile: Mail URL Length
- Mobile: Java version
- Mobile: Manufacturer
- Technology: Connection Types
- Technology: Monitor Color Depth
- Technology: Monitor Resolutions
- Visitor Profile: Top-Level Domains
- Visitor Profile: Domains
- Visitor Profile: Geosegmentation
- Traffic Sources: All Search Page Ranking
- Traffic Sources: Original Referring Domains
- Custom Variable: s.server report
For the most part, the information in the separate reports can determined just by knowing the device (which is also a default Omniture report). So, a single report can take the place of 10.
There’s also the pesky issue that the reports more often than not show “Unknown” for 90%+ of the mobile traffic (at least, in the U.S.). So not only can the data be determined from knowing the mobile device being used, the additional reports aren’t even well populated.
The “Connection Type” report, along with “Monitor Color Depth”, measure things that haven’t been an issue in too many years to continue reporting on. LAN, 16-bit or higher.
“Monitor resolution” is irrelevant in the face of also having “Browser Width” & “Browser Height” reports (the true size of the web page “real estate” on screen).
Visitor Profile reports
My dislike of the identified Visitor Profile reports are due to halfway implementation. The “GeoSegmentation report shows a nice map representation, but only of traffic metrics like Page Views and Visits. Why not open this up to conversion variables and really make the visualization useful, instead of needing to rely on the “flat”, non-map Visitor Zip (
For the “Domains” and “Top-Level Domains” report, you have granularity issues; the “Top-Level Domains” report is sort-of a country-level report, but the U.S. has several line items. The “Domains” report shows what ISP people are using to access the Internet (which I think is generally useless in itself), but again…it spans geography, so the ISP network someone is on may not even have the same technology. So what are we really measuring in these reports?
Traffic Sources reports
The “All Search Page Ranking” report seems like it could be useful, until you realize that 1) it aggregates all search engines (whose different algorithms provide different rankings and 2) with personalized search, rankings are no longer static. Literally every single person could see a different link position for the same search term. So while this report may have made sense for SEO measurement in the past, it’s really past it’s prime…use the right SEO tool for the job (Conductor, SEOmoz, and the like).
The “Original Referring Domains” report is weird in its own way…the absolute first URL that referred you to the site. Really? As Avinash has said, giving 100% credit to the first touchpoint is like giving your first girlfriend credit for you marrying your wife (paraphrased). This report is very limited in its usefulness IMO, especially given the advances in attribution modeling in the past several years.
Custom Variable: s.server report
The only custom variable report I have on this list is the
s.server report; hopefully, all of your other custom variables are capturing only business-useful information!
The reason I dislike the
s.server variable/report is the same reason I dislike the “All Search Page Ranking” report; use the right tool for the job. This is a lazy way of monitoring server volume for load balancing. But if you’re doing the job well on the back-end, shouldn’t every server have the same level of volume?
Even if the answer to the previous question is no (I’m not a network engineer, clearly), having an operational report like this doesn’t make much sense to me in a marketing reporting tool.
By hiding reports in the Omniture menu interface, this doesn’t mean the info stops being collected or becomes unavailable to all users. Rather, the option to use the reports isn’t immediately obvious (since they don’t show up in the menu). Power Users can still find these reports using the search box if necessary to answer an oddball question.
But in my experience, the information in these reports are generally not business useful, or are lacking in some critical way. If you can’t make regular, high impact decisions with the info, then you’re better off never looking at it at all.