Fifteen years later, I’m writing about a new big splash for EMC in the SRM space – ProSphere 1.0. I’ll stop you right here and tell you that you need to go read Chuck’s post on the product. I can’t out-do his deep-dive into the industry angles and why it’s such a big deal, so I won’t even try.

What I will tell you is why working on this product was so different from any other product I’ve touched at EMC, and why I’m so proud to be able to announce it here. Just like fifteen years ago, it was a chance to take a look at everything “wrong” while also still looking in new directions at the same time the industry is making another scale leap with Cloud environments. This has been some of the hardest work I’ve done here at EMC. But seeing it get out the door is making it all worth it.

EMC ControlCenter has a decade-plus roller-coaster history.  It’s been through some challenging times, on many fronts, but we’ve never stopped trying to improve it for our customers.  We often talked about things we’d like to do differently, given the chance.  By the middle of 2008, a vision for the future was firming up – and we had a name for it.  “SRM 7,”  reflecting our goal to avoid “growing” ControlCenter to a new major version after 6.x.

We handed out T-shirts with a stylized “7” inside a diamond shape (which might evoke a certain superhero).  The (perhaps dangerous) implication?  SRM 7 was going to fly in and save the day.  We were all a bit skeptical.  But today we’re announcing the first shipping release of that product – ProSphere 1.0.

Why do I think it’s such a big deal?

We didn’t rebuild ControlCenter

Early on, we faced a critical decision – do we rebuild ControlCenter piece by piece, or do we build a new solution from the ground up?  We knew part of the issue with ControlCenter was feature creep.  We wanted to focus on critical customer use cases and build the application that could do those, and resist the temptation to build a giant unwieldy Swiss Army Knife.  That philosophy bled into everything.  We didn’t build a giant infrastructure that could address all our needs; we architected an extensible solution and implemented enough of it to get us through the use cases we were attacking.  We avoided writing “just in case” code to support possible future features.  We didn’t build what we could reuse.  We knew we couldn’t ever finish this in time if we wrote it all from scratch, so we pulled in proven components from other shipping EMC products and integrated them.

Further than that, we didn’t want to be the single best interface for every deep use case – we knew you wanted to use the right tools for those jobs.  So we made a plan — find those tools, link to them, launch them in context, and make your sign-in transparent – and in the process eliminate thousands of lines of code which need to be tested, debugged, upgraded, and so on.  It’s a win on every front.

You’re never going to have a “lean” piece of software to do the giant job of managing the storage for your entire enterprise.  But ProSphere is downright svelte compared to ControlCenter, and we intend to keep it that way.

It was ok to look outside

Another shift we made in ProSphere was to look outside our traditional sources of software components.  We didn’t want to write, maintain, and test unneeded software.  We didn’t want to architect and design unneeded components.  We pulled in open source software, we used open standards, and we got creative.  It was a learning curve for the development team, but in the end we have a product that communicates using known web technologies and patterns, and which we hope will serve as the foundation for an open, extensible management solution.

This extends deep into the product’s DNA, not just a superficial claim about what our APIs look like.  Eliminating traditional agents, using HTTP to communicate between our machines, going to a virtual appliance model, using industry-standard system monitoring  … all these things make the system higher quality and more extensible with less code to carry.

We got Agile

One of our early decisions was to abandon the waterfall software method we had more or less down to a science in ControlCenter, and replace it with an enterprise-scale Agile development approach.  In my personal opinion, two excellent things came out of that decision: we “forced” our development managers to take ownership of use cases within the product, and we created cross-functional (development, quality engineering, documentation, product management, user experience design) teams to tackle those use cases in close collaboration with each other.  In my fifteen years of software development, this is the closest cooperation I’ve seen between functions on a project team of this scale.

We built a safety net

Last, but most certainly not least, this product has the most aggressive safety net I’ve encountered.  Every day, thousands of automated tests run against the system in various forms – unit tests against the code, integration tests against the REST interfaces, automated UI tests against the finished product in a test environment (deployed automatically after every finished build), and even more automated UI tests against the finished product in a “real” environment.  Individual developers had access to a simple web-based tool to deploy a build, patch it, and run a suite of hundreds of automated regressions against it prior to delivery – all from their desks, without ever seeing an installation screen.

We combined this with static analysis tools constantly analyzing the source trees to check for bugs, security lapses, stylistic violations, and undesirable complexity, with web-based dashboards for everyone to see.  Finally, there’s a dedicated team doing manual run-throughs of customer use cases in a variety of real-world environments.  You take all that and combine it with an organizational mandate not to tolerate technical debt, not to tolerate little bugs accumulating in the system, and you’ve got a team that understands what quality means and has an extremely low rate of regressions.

Part of what took us so long was building all that scaffolding, and changing the culture of our development and test organizations to get us here.  And now that it’s built (not that it’s ever “done!”), it will pay for itself for years to come.

In conclusion

I’ve been involved on this product for about three years now … the same amount of time I’ve been a father.  And like any Proud Papa, I can’t wait to show off.

We built a solid foundation – a highly deployable, serviceable, and usable application – and concentrated our efforts on a small, tight family of use cases using that foundation.  We’re already working on what’s next.  I think we’ve changed the game here.  I’m proud to be a part of that.  I was just one manager, with one small team, working on little bits and pieces of this giant project.  I’m grateful to my team for consistently seeing the big picture and working nonstop to get us there.  I can’t begin to explain how proud I am of the work they did.

It hasn’t been easy; a lot of blood, sweat, and tears are flowing in the hallways.  But nothing worth achieving is easy, and there are a lot of smiles today as we finally take the wraps off this.

ProSphere 1.0 is just the first step.  It’s your turn now.  Check it out.  Tell us what we’re doing right, what we’re doing wrong.  Help us take this to the next level.  I promise I’ll listen and do my part to make sure your voice is heard.

This post is from: Dave Talks Shop

Introducing ProSphere

One of them was something I learned from Steve Todd, way back in the day — always excel at your day job before you do anything else at work.  Doing a great job in your primary responsibilities is what gives you the freedom to explore secondary responsibilities.

I’ve been eyeball-deep in work on EMC’s next offering for Storage Resource Management (what you may have heard referred to as “SRM7″).  Never mind my days, it’s consumed my nights as well!  So I’ve let this blog get a bit rusty.

There are some other rules I’ve set for myself, and it’s cool to see that the “social” folks at EMC have codified some of those rules into a little video that people here actually get training credit for watching.  It’s short and sweet, and I was pleasantly surprised.

This post is from: Dave Talks Shop

Rules for blogging here

Good luck to any applicants!

This post is from: Dave Talks Shop

Cool job opportunity at EMC

I don’t know yet if I’ll be there, but I can tell you what will be there — a handful of sessions describing the new storage resource product that’s been occupying all my time of late.  There’s even a hands-on!

I can’t yet tell you the product’s marketing name (pesky lawyers), but we’ve been calling it “SRM 7″ internally for a while now, and we’re wrapping up the first round of beta sites and pushing full speed ahead for a coming general availability release.

If you’re a ControlCenter customer and want to see what’s coming soon, make sure to sign up for any/all of the “Beyond ControlCenter” sessions listed in the EMC World course catalog.  Tell them Dave sent you .

This post is from: Dave Talks Shop

Beyond ControlCenter

I remember the day very well … because it was my first day of “real work,” and it was a snow day.

Without any real working-from-home infrastructure to speak of, I didn’t get an email about this at work, and I didn’t spend the day online from home.  I got up in the morning and checked the news, and it was listed that our Westboro and Southboro facilities were closed to nonessential personnel.  I don’t know if I’ve ever seen that again since then – but I know I thoroughly enjoyed having my first day as a salaried employee being spent inside watching the snow pile up.  (Don’t get me wrong, I love my job, but being paid to watch it snow is a pretty good deal, especially at 22…).

At the time, of course, I wasn’t working for EMC.  I had never heard of EMC.  I was working for Data General.  I was still a graduate student at WPI; I had completed one semester as a Teaching Assistant, and took the job and did the rest of my coursework part-time.   My classmates and professors thought I was mad – the future was not minicomputers; why would I go to work for a dead-end place like DG?  They weren’t pushing for EMC either of course; they were excited about the Internet, they were excited about consulting companies, about MathWorks, and so on.  The future wasn’t on 495, it was inside 128.

Well, I was happy on 495.  I was thrilled to be part of the CLARiiON team.

Turns out it was a good plan!

A lot has changed in 15 years.  A lot has changed in the overall industry, in the Massachusetts tech scene, and certainly here at EMC (and DG!).   I had high hopes of putting together a post full of things that are different now than they were in 1996, but frankly I don’t know where to start, and I’ve spent the last few nights working on “real work” (now that’s a change from 15 years back!) and couldn’t really do the necessary research.

But I will share the closest press release I could find to my joining date: this one, from DG .  It announced the joint creation of a 2.5 terabyte DSS in Tokyo.    2.5 TB … the amount of disk space I have on my desk.  About a month after I joined, DG announced a new disk array with capacity of 84 GB per chassis.  The Blu-Ray XL format, standardized last year, puts more than that on a single optical disk.

A lot more than just capacity numbers has changed.  The name of the corporation that pays the bills, the technology I use every day, the way I manage my work/life balance, my career goals … all are a lot different than what I signed up for when I took that job offer.  But what hasn’t changed is that I come in every day challenged by exciting problems, and I try to leave every day knowing I helped my co-workers to solve those problems in a good way.  There are worse ways to make a living … much worse.

Thanks EMC, thanks DG, thanks to the crew that hired me (you know who you are), many of whom are still here — and many thanks to all the people I’ve worked with since then.  And I couldn’t do it without the loving support of my family; I love my job, but coming home every day is what makes it all worthwhile.

I wonder what I’ll be looking back on in 2026….

This post is from: Dave Talks Shop

Fifteen years!

Perhaps most exciting, if you’re a ControlCenter customer, is the chance to get an up-close look at our next major release, what we’re calling “SRM 7″ internally.  There’s going to be a dedicated system on the show floor manned by some overworked and underpaid colleagues of mine, who will walk you through some simple use cases using our latest build of our in-development product. Please take the opportunity to grill them and provide some feedback. We’re all marching full speed ahead on this product and some real customer feedback at this stage is going to be incredibly useful to us.

There’s also some lecture/demo sessions set up, including “Next Generation Ionix ControlCenter v7 is Here…What’s Next?” and “Achieving Time to Value with Next Generation SRM: An Architectural Perspective.“  Both of these will take you behind the scenes with where ControlCenter is headed.  I wish I could see them, and more than that, I wish I could hear what you the customer are saying at them.  Feel free to reply to this post or hit up the ControlCenter customer community with feedback; I promise it’ll get seen, and not just by me.

And if you’re more interested in adding value to your current ControlCenter install, don’t forget the always-popular hands-on sessions and a special demo/lecture on “Ionix ControlCenter v6 – Under the Covers.”  I imagine that one’s going to be pretty informative.

There’s a ton of other Ionix presence scheduled, including some hands-on work with Storage Configuration Advisor (the “next-gen” compliance product that got some sneak-peaks last year) and a slew of vBlock/Unified Management presentations.  It looks like quite a full menu for the Ionix fan.

This post is from: Dave Talks Shop

Ionix at EMC World

Fortunately, I didn’t toss it too far aside.

If there’s one thing I’ve learned in almost 15 years at EMC it’s that change isn’t disruptive to the status quo, it is the status quo.

Everything I said I was doing before, I’m still doing.  I’m wearing a lot of hats right now.  My small development team has grown as it takes on more responsibility, and I find myself playing the roles of Scrum Master, Technical Lead, and Development Manager.  Somewhere in all that I’m trying to individually contribute technically as well, but that is at the bottom of the list.

The other thing that keeps falling off the list is contributing to “the conversation” (both internally at EMC and externally on twitter and in people’s blogs).  I’m afraid that is going to be an uncomfortable reality while I try to wrap my arms around all these roles and make sure my own commitments aren’t being missed. Try not to do too much without me .

It’s a great place to be, in the thick of the action, surrounded by good people.  I’m never bored, I’ll say that much!

This post is from: Dave Talks Shop

Personal update: putting new hats on

The concept: write a project where multiple EMC developer technologies can be used in a single program.  Bonus points for incorporating other online technologies.  Win money and fame and the adoration of the world.

I’m waiting for the detailed T&C, but you can read up more on it here.  Innovation through contest is something EMC has tinkered with quite a bit, as you may have read on Steve Todd’s blog last year.

Definitely check out the link for more info. I’m hoping I can find some time in between all my “real work” to put a couple of these tools through their paces.

This post is from: Dave Talks Shop

EDN puts its money where its mouth is

So here, as requested by Natalie, is my version of the web at 20…

How has the web changed my life?

It’s a bit of a cheat for me to answer this question, because what truly changed my life were the networks that predated what we think of as the Web.  The web made them easier to use and broadened their scope by orders of magnitude, but the damage was already done.

I would not be where I am in life without the Internet.  As a teenager, I hungrily sought out information from any source I could.  On a TRS-80 Color Computer hooked up to a tiny black&white TV, at 300 baud, I connected to (and eventually ran) bulletin boards, snuck into unprotected university dialins to play MUDs and read Usenet, and connected to individuals and information from a much bigger world.  I am still in contact with some of those people, still use the behaviors I learned back then every day.

But it wasn’t until college, in 1993, that I saw those things melded together into The Web.  It may have been technically 3 years old by then, but it was just getting its momentum.  It became my immediate and constant companion, and has been since.  Everything I cherished about the Internet was boiled down into one magical term: Home Page.  We didn’t have net connections in our dorms, so all that gweeping was done in the semi-dark basements of the CS building, in labs shared with giant line printers and dozens of black and white monitors.

I can still taste the Mountain Dew … and the freedom.

I created my first web page in those years, when the best web search engine was called WebCrawler and people still coded for users of Lynx.  The Internet Archive has a version of that page, from right before I graduated.  Most of the links are incredibly broken, but you can still see a snapshot of my personality in the text, personal branding way before the Millennials “discovered” it.

This was the brave new world.  And we thought we’d keep it to ourselves forever: nerds arguing over Star Trek and D&D, posting pictures of our cats, and researching new technologies.

And then some damn fool figured out how to make money off it all.

How has the web changed business and society?

I like to say the changes to society and business associated with the web have come (and are coming) in waves.  For a long time, businesses saw the web as nothing but a giant Yellow Pages, and society saw the web as a place to argue over Star Trek.  I remember a magical period in the Web’s history when business hadn’t caught on yet, but there were enough people for actual connections to be made.  You could find people who had been to far away places and talk to them about their experiences.  You could bump into groups who were dedicated to obscure programming languages and figure out how to solve bizarre software problems.

And then the marketers took over, clumsily but powerfully. When you tried to find real people, you found storefronts instead.  Search engines were new, and SEO technology outpaced the search algorithms.  You couldn’t trust the web any more.  Communities were buried, hard to find.  It was difficult to meet new people and form new interactions.

Eventually the old sense of community emerged from its hibernation.  Strong web forums with passionate moderators helped people with similar interests hook up, and some of them lasted long enough to become trusted sources of information.  Social media sites formed and helped us track trusted crowds.  Web page technology got decent, bandwidth got cheap, blogs became mainstream, and suddenly (if you knew where to look) the web was social again.  Now there were two webs, the social web and the static clumsy business web.

Then, most recently, businesses figured out how to leverage the new (old) online world.  Instead of trying to take over, they tried to engage.  And the business web became social.

And so we’re back where we started, but better.  We’re free to argue over Star Trek and post pictures of our cats … and route around government censorship … and collaborate on new technologies … and tallk directly to our government … and order pizza online … and monitor millions of conversations until you find an unhappy customer in Paraguay … and finally engage with that person following the same unwritten rules we geeks help put into place 20 years ago.

It’s a beautiful time to be an information professional.

(What I think is an important followup point here is that there are areas where the web hasn’t changed society.  Vast stretches of people are not connected, and the disconnect isn’t shrinking.  Let’s not forget this.)

What do I think the web will look like in twenty years?

To answer this I tried to think back on the past twenty years.  Many of the technologies existed when the “web” was born, but we found innovative ways of tying them together, made bandwidth cheaper, and exponentially extended its reach.  So what do we have the technology to do now, but aren’t doing yet?  What will change when (and if) the digital divide narrows?

The easiest answer is that dumb search will disappear.  All search will be contextual by default, whether that context is geographical, social, historical, or something else we haven’t thought of yet.  Our tools will serve us, help us filter the world automatically in contexts that make sense to us.  Based on aggregating data about ourselves, our histories, and our friends, the tools will be highly predictive and accurate.  They’ll work on objects other than text (we’re improving image search, but let’s imagine all of youtube indexed not by metadata but by the data itself!).

This will come at the cost of privacy, of course, and the mad scientists of the 2030s will be those who refuse to make that trade.  Like a person in today’s world who refuses to have a credit card or a bank account, most of us won’t be able to understand how they can reject all that convenience.

Another easy one is that we’ll take the cloud for granted.  If you have data somewhere, you’ll have that data everywhere.  The concept of remembering a URL or bookmarking it and losing that bookmark will seem archaic.  There’s some fascinating security and usability problems to be solved there, of course.  It’ll be fun to see that fall into place.

Fads will come and go faster than they do today.  With the ability to spin up a virtual data center and tear it down with no delay, a startup can flare up and disappear within hours.  Low-budget clones of such companies will appear worldwide, and the battles over who had an idea first will be epic.

Another trend I think will continue is the shrinking of content.  Real writers will be harder to find, as the majority of content providers end up doing nothing but sharing links and snippets.  Our attention spans will shrink further.  If we can’t read it or watch it in 30 seconds, we won’t care.  And the few of us who insist that things used to be better will be laughed at by our juniors.

One thing I can predict is that twenty years from now, my daughter will 21 years old, and she will laugh uproariously at how wrong we all are about where things are headed.

The more things change, the more they stay the same.

I’ll ask Jamie Pappas, a colleague at EMC, to continue the discussion next.  Jamie?

This post is from: Dave Talks Shop

The web at #20years old

I won’t give a play-by-play of the event, but I will say it was energizing to see so many people so excited about bringing new ideas to EMC. It’s always humbling and inspiring to sit among intensely intelligent people who speak with confidence, poise, and command about complex ideas.  We spend so much time specializing in our day jobs — getting really good at doing the jobs assigned to us — that it’s easy to forget the rich backdrop of brainpower this is all built atop.

I had a chance to catch up a bit with some old colleagues, put faces to some names, and connect with some people I hadn’t met before.  The conference took place around the world, with one hour shared at all sites through videoconferencing, when the new class of Distinguished Engineers and Fellows was announced along with the awarding of prizes for top innovation ideas.  I was sitting next to one of the submitters of the top prize-winning idea, and it was great to see him receiving accolades and congratulations through the rest of the day.

The conference had a few speakers from outside EMC, including Dave Ritter from Innocentive.  They provide a framework for crowd-sourcing problem-solving, and Ritter had some fascinating insights into innovative thinking.  The notes I scribbled related to boundary objects and problem statements.  He echoed a lot of what I feel about diversity and effectiveness, and I look forward to reading more about what he and Innocentive are doing.

We also heard from a handful of executives during the conference, and it was refreshing to hear some of their no-spin commentary on where EMC is, what the company needs from us, and where it’s headed.  As something of a cynic, I expect to be suppressing a bit of eye-rolling during executive presentations, but they avoided the temptation to drown us in Kool-Aid and instead gave us some real insight into the business.

The event was a success on multiple levels, and I’m glad I could make time to attend.  The real question is how to make sure the spirit of innovation and engagement I saw everyone displaying gets spread throughout the company the remainder of the year.

This post is from: Dave Talks Shop

Hanging out with the braintrust