My Summer At Sun

I’m home again after my time in Boston with Project Aura. As I look back over my summer I am certainly nowhere near where I expected to be at the outset.

One part of the difference is simply that once I took the concrete step of saying “I don’t like robots and am going to actively pursue finding something I really do like,” it shifted how I see the world significantly. That process has largely been internal, but the external experience of working for Sun and being an intern was also significantly different than I expected it to be.

HR sent around an intern survey to ask us about how much we enjoyed our work experience and how we would rate the organization and whatnot. I filled in the blanks and was not looking to make any serious commentary. At the end, however, the survey wouldn’t let me finish without putting something into the blank for “What would you recommend that management do to make this a better place to work?”

The problem is the question is not a simple one to answer, so as is my wont, I gave them likely far more information than they wanted:

The balance between individual expression and managerial expectations was ill-defined at times. I was given what I took to be a relatively large amount of freedom in the type of solution that I would design to solve a problem, but when I completed the process and went back for additional feedback, I found that there had been a more specific task expectation that was not explicitly stated.

While, ultimately, the reasons for the specificity did make sense, discovering the boundaries was in large part a process of trial and error rather than explicit definition.

This is largely, however, a particular criticism of my intern experience and not necessarily a generalizable statement about work at Sun. I had, frankly, what I now recognize to be unreasonable expectations of the level of autonomy and creativity both given the development timetables for the project and my own role as a short-time member of the team.

In short, it took me a while to come to terms with being “just an intern” after several years working independently and designing systems according my ideals and expectations. I don’t question my work overall, but the types of designs I initially attempted were much better suited for projects at least a year from launch rather than several months. There are still innovations there that I think could benefit the field overall, but they are non-central to the goals of my particular project to an extent it is quite reasonable to not include them. It just took me a bit to figure out those boundaries.

Are there specific management practices that I would change? Honestly, no. I have been very satisfied with my experience here. I began this summer hoping to understand the field of music recommendations as well as corporate culture better, and at the end of three months, I feel Sun has been very helpful with both of these goals. That it took me a while to understand my role I don’t see as a failing on the part of Sun or myself, but simply as a necessary process of job and worker coming to a consensus on the role to be played.

The pattern for how much summer went is pretty straightforward. I had some code monkey-type tasks like system configuration or debugging, and I did those pretty well. Then I had some design-type tasks and I did those about two orders of magnitude too complex. It wasn’t intentional, I simply tackled the problems focusing on coming up with the best solution I could come up with, without considering the very reasonable requirement that the solution be feasible within the project timelines.

  • Week Two Task: “Build a Collaborative Filter”

    • What Was Expected: Implement Amazon’s Item:Item filtering algorithm.
    • What I Did: Constructed a massive conceptual structure based on semiotics and Markov models that would allow the integration of a variety of information sources into a unified structure.
  • Week Five Task: “Compare TF-IDF to Item:Item Similarity”

    • What Was Expected: Implement Amazon’s Item:Item filtering algorithm in Java using the Aura datastore.
    • What I Did: An extensive examination of the algorithm and dataset including an implementation in SQL. This particular deviance is a bit more debatable because the analysis essentially revealed that the research question being asked can’t be answered with the dataset that was being used for testing.
  • Week Ten Task: “Implement Item:Item Similarity Using the Aura Datastore”

    • What Was Expected: Use existing offline methods of offline computation to compute item:item similarity to be updated on a daily or weekly basis.
    • What I Did: Designed a hybrid DISC computing model for dealing with structured data to allow similarity generation and maintenance in real time.

I don’t think that the ideas I had were wrong per se, but I do now understand, as I said in my review: Aura hopes to release in a month or two, and I’m an intern for three months. Grand schemes take time and are prone to failure as unexpected issues arise. There’s a balance that has to be struck in just how big of a problem to tackle and my attempts frequently missed that mark.

Ultimately I spent too much time trying to adapt the system to my expectations. I was however happy with my final piece of work: iterative file-based computation of item:item filtering. It implements a known algorithm in a way that minimizes computations and allows efficient parallelization without shared memory.

Leave a Reply

Your email address will not be published. Required fields are marked *