Project Phoenix Week Open Mike: Michael Hughes Delves Deeper into the Products & Services Survey

STC President Michael Hughes brings Project Phoenix Week to an unofficial close with Open Mike. In this post, he provides a more in-depth look at the results of the recent Products & Services Survey and provides a few tables and graphs of interesting information.

On 30 November, we posted the summary tables from our Products and Services Survey (available here in PDF format). I wanted you to have the same data we had so you could start to form your own opinions. I have gone through the rather dense, data-rich tables and have done some preliminary analysis that gives me a clearer view of the messages within the data. I want to share some of the things that have emerged for me.

The Intersection of Importance and Quality

There were a series of questions that asked you to look at specific services or products, first along the dimension of how important they were to you, and then along the dimension of how you rated their quality. It was interesting to compare the importance and quality ratings side by side. Table 1 shows the average ratings for the list of services and features of STC.org for their importance and quality (on a scale of 1–5). It also shows how the services ranked among each other within each dimension.

Table 1: STC.org services/features (questions 3 and 4)

Service/Feature Importance
Avg.
Importance
Rank
Quality
Avg.
Quality
Rank
Intercom Magazine 3.68 1 4.14 1
Salary Database 3.52 2 3.82 3
Career Center 3.15 3 3.57 6
TC Journal 3.14 4 3.96 2
Conference Website 3.03 5 3.65 5
Membership Directory 2.83 6 3.57 6
STC Notebook 2.62 7 3.67 4
Community Admin. 2.45 8 3.32 10
Job Seeker Boot Camp 2.43 9 3.54 8
Buyers' Guide 1.79 10 3.5 9

 

Figure 1 shows that Intercom, our salary database, career center, the Journal, and the conference website all landed in the quadrant of above neutral (3) in importance and above neutral (3) in quality. This quadrant captures our strong points, things we must be careful not to break as we move forward. The membership directory, STC's Notebook, community administration, job seeker boot camp, and the Buyers’ Guide fell in the quadrant of “not as important but doing well.”

None of the features or services fell in the quadrant of “things that are important but which we aren’t doing well” (above 3 in importance but below 3 in quality). That quadrant usually sets off alarms akin to the Lost in Space robot flailing his arms and yelling, “Warning, warning!”

But we certainly want to improve our quality ratings in all of these areas, and this survey gave us some benchmarks against which we can measure our progress.

Figure 1: Quadrant analysis for questions 3 and 4 


Table 2 compares the questions that dealt specifically with our education services.

Table 2: Education services (questions 8 and 9)

Service/Feature Importance
Avg.
Importance
Rank
Quality
Avg.
Quality
Rank
Live Web Seminars 3.45 1 3.61 2
Archived Web Seminars 3.42 2 3.44 6
Summit@aClick 3.28 3 3.59 3
Conference Proceedings 3.23 4 3.52 4
Online Certificate Courses 3.14 5 3.66 1
Academic Database 2.74 6 3.49 5

 

Figure 2 shows that our live web seminars, archived web seminars, Summit@aClick, conference proceedings, and online certificate courses all fall into the quadrant of above neutral importance and above neutral quality. Our academic database is in the quadrant of “not so important but doing well.” Keep in mind that if you are looking to go back to school to advance your professional development, you probably see the academic database as more important than this average rating.

Figure 2: Quadrant analysis for questions 8 and 9


 
These results give us some strong benchmarks to measure our progress against, and help us set our priorities of what features and services to concentrate on.

Publications

Figure 3 shows how our members read the two STC publications. I find it interesting that they have different profiles. Not quite sure what to make of that yet, but I wanted to share the picture with you.

Figure 3: How members read the publications


 As Liz Pohland indicated in her recent blog, we are looking at various options with our publications. Figure 4 shows how members felt about some of those options (along a seven-point scale from “I would like this” to “I would not like this”) What I find most informative is that the last two options, making Intercom free for nonmembers and offering Intercom online only, dominated the negative end of the spectrum.

Figure 4: Opinions about possible publication strategies

Future Development

We also asked you to rate where you wanted to see us focusing our efforts going forward. Table 3 shows how you responded with the choices in descending order of importance.

Table 3: Future development areas

Area Average
Education and Professional Development 4.18
Career Services 3.71
Networking 3.79
Publications 3.65
Affinity Programs (i.e., Member Discounts) 3.16

Qualitative Comments

I am still wading through the qualitative comments (over 550) and I am finding them to be quite insightful as well. I will report on my perspectives there in a later blog.

Thanks to all who participated in this survey. The staff and the board take your input seriously.

5 Replies to “Project Phoenix Week Open Mike: Michael Hughes Delves Deeper into the Products & Services Survey”

  1. An excellent snapshot of members’ desires! As a former board member and STC President, I can see the value of this information and I know that the current board will do all it possibly can to work with the information to provide requisite information and services.

    Thanks, Mike, for analyzing it all for us!

    1. Bill, the question Table 1 is based upon simply asked, “Please rate the importance of the following features/services of http://www.stc.org to you.” Table 3, however, is from a question that asked about future importance: “Going forward, how important is it to you for STC to develop new products and services in each of the following areas?”

  2. When I worked in radio, I learned a very important thing about the ratings: They- answer only a part of the question. Basically, a survey of this sort only tells you about the relative value of what you are currently doing. I doesn’t, and can’t, answer the question, “what if we did…. ?”

    The result, staying with the radio analogy for a moment, is that radio stations often end up removing the least well-performing bits of programming, and next survey, they do the same, and on and on, until you get the kind of homogenized, uniform, unexciting radio that dominates the airwaves today.

    Let’s be sure, going forward, that we do serious, critical analysis of the type that Mike has just given us, so that we can improve our current offering, and maybe also remove things we thought were good ideas but the membership doesn’t want or need.

    Let’s also be sure, going forward, that we continue to think “out of the box,” that we use Edward de Bono’s ideas of creativity to generate new ideas, new services, new ways to organize ourselves, that maybe nobody ever thought of – and that we give them the chance to succeed.

    Giving new ideas the chance to succeed means putting them into action, and LEAVING them there long enough to know if they’re working, without hanging on to them once it becomes obvious that they don’t. This needs a combination of conviction (about new ideas) and humility (about observing the results), with the ability to admit mistakes, correct them, and move on.

Leave a Reply