Hawthorne Effect

The Hawthorne Effect is something I've seen numerousness times on projects.  When auditing or introducing a new process, do you tend to see people doing a better job than you expected? The Hawthorne Effect refers to the tendency of some people to work harder and perform better when they are participants in an experiment. Individuals may change their behavior due to the attention they are receiving from researchers, auditors, or coaches.

This effect was first discovered and named by researchers at Harvard University who were studying the relationship between productivity and work environment. Researchers conducted these experiments at the Hawthorne Works plant of Western Electric. The study was originally commissioned to determine if increasing or decreasing the amount of light workers received increased or decreased worker productivity. The researchers found that productivity temporarily increased, regardless if the light was increased or decreases. They then realized the increase in productivity was due to the attention given the workers by the research team and not because of changes to the experimental variable.

Do you audit your processes?  How do you ensure you get a true representation of project efficiencies and not suffer from the Hawthorne Effect?

HT: Wikipedia Like the drawing? Get it at Pictofigo

Describing the SDLC

SDLC

While assisting an IT department through a Sarbanes-Oxley (SOX) audit, I had to document an organization's Systems Development Life Cycle (SDLC). The SDLC includes activities and functions that systems developers typically perform, regardless of how those activities and functions fit into a particular methodology.  Many assume SDLC is referring to a software development process.  In turn, there's a lot of debate about different development practices and approaches.  For example, when I lead Scrum teams for an organization, as part of an overall SDLC, all of the Scrum activities took place during the Implementation phase.  When changes were deployed to the Production environment, the Support team leveraged Kanban.  From Planning to Analyzing to Designing, they leveraged a Waterfall process.  It all began with a request for a change. Because a picture is worth a thousand words, Pictofigo has created a SDLC poster, with a little input from me. You can either purchase it from CafePress as a poster or you can download it from the Premium Pictofigo site.

Meeting with GAO

After finding out the Government Accountability Office (GAO) was coming to pay our program a visit, I was also told to work with a small cross-functional team to collect all of the data to meet their requests.  There was a list of recommended executive actions and we had to prove how we were satisfying those recommendations.  To visualize our progress of collecting the data, I used a physical task board and sticky notes.  I would call it a Kanban but we really didn't have any work in progress (WIP) limitations.  The board was comprised of 4 columns:  Backlog, WIP, Blocked, Done.

As of last night, everything was in the done column and I even had one team member come up and shake my hand.  For some reason, I think there may have been a lack of confidence that we could identify and collect the data requested.  With some leadership, inspiration and clear goals, we got it done.  Though I'm not at liberty to say exactly what we supplied them, the requests they made were not unreasonable.

I've been through a SOX audit before so I understood how an audit works.  Provide proof that you do what you say you do.  Be able to explain why you do it.  Now, what you do and how it aligns with how others think you should do it is another story.  But, if the auditor is not satisfied with how you do it, they will make a recommendation on how you can meet their expectations.  Here is the important thing.  An auditor does not care what you say you are going to do.  They care what you say you've done or do.

PMP Application Process and Reading Instructions

PMI Audit

An interesting thing happened when I was approached by a coworker asking for assistance in completing his PMP application.  His concern was he would be audited and word around the office was that I had been audited by PMI and did just fine.  Both facts were true and I was more than happy to assure him that as long as he was factual about what he put in this application, he would have nothing to worry about.  All the same, he said he would feel more comfortable if I would review what he wrote. He admitted that what he was going to show me wasn't his actual application, but rather, he was working on a spreadsheet to make sure he had his bases covered.  Good idea, I said, show me what you have. What he brought up was an Excel workbook, provided by another coworker who had recently attended a PMP boot camp. I noticed right away that he had 7 projects listed totaling over 10,000 hours.  What really caught my attention was the breakdown of hours across the process groups. Below is an example of one of the projects.  Names and titles have been changed but the hours were not.

Company

Project Title

Job Title

Start - End

Total Hrs

Initiate

Plan

Execute

Control

Close

Acme

Foo

Sr. PM

11/04 11/05

2000

100

500

1000

300

100

I asked him how he came to such exact amounts per process group.  He pointed to help text listed in the bootcamp-provided workbook.

In the following worksheet, we will try to compile the hours you spent on each Process Group for each of your project. Eventually, you will need these hours to fill in the PMP Application... Let us assume that this is the typical project manager job with about 85% of the hours involving tasks similar to those in the Typical PM Task worksheet.  Assuming this was a typical project manager task with a rough distribution of 5% of these hours in Initiation, 25% of hours in Planning, 50% of hours in Executing, 15% of hours in Controlling, and the remaining 5% in Closing.

When I asked him to explain what I was seeing, he stated he had worked on the project for a year and took a 2 week vacation.  (2,000 hours).  Then he said something that surprised me.  Well, the worksheet said to put percentages in so that's what I'm doing. My response was "Noooooo, that's for a "typical" PM doing "typical" PM tasks. I told him was listed in the help text was a calculated average and only there as a guideline.    In addition, I clarified, I didn't think he could map 100% of his time to deliverables.

I told him that he needs to look at mapping his work experience to the process groups like he would if he were identifying and scheduling activities of any project. (Reference the PMBOK for items in blue)

• Define Scope (Section 5.2 - Page 112)

• Create Work Breakdown Structure (WBS) (Section 5.3 - Page 116)

• Define Activities (Section 6.1 - Page 133)

• Sequence Activities (Section 6.2 Page 136)

• Estimate Activity Durations (Section 6.4 - Page 146)

Now, you may disagree with what I proposed but I hope you understand my frustration after seeing something like this.  First, he didn't read the instructions correctly.  Second, his primary concern was being audited, not meeting the fundamental requirement of detailing work activities.   This brings me to my final point.  Before defining scope, you need to collect requirements(Section 5.1 - Page 105).

I've come to find out, he doesn't even want to get the PMP. He's being pressured by his company to get the certification.  The whole situation could be fodder to half a dozen posts or articles.  I don't know what part bothers me more, that he missed the assumptions listed by the PMP boot camp or the fact that his company is pressuring him to get a certification that he doesn't want.

I wish him luck.

Judging PMI Unfairly

Recently, I've been a bit vocal about my concerns of the volume of people getting the  Project Management Professional (PMP®) certification.  I often ask if quality is being sacrificed in order to certify more people.  Don't get me wrong, if being a PMP means you are a "quality" project manager, I think the more the merrier.  But, what if more PMPs means the Project Management Institute (PMI) doesn't have the necessary people to conduct proper audits?  I decided this question warranted a telephone call to PMI.  Though they would not give specific numbers, they stated

The number of audits have remained consistent over the last few years

Ah-ha!  Just the incriminating statement I was expecting, to support my theory!  Clearly the increased rate of people getting certified would mean a lower percentage were being audited.  Since numbers don't lie, I went back as far as I could and compiled the monthly PMP certification rate.  September 2006 was the earliest (easily accessible) date I could find.

Once I charted all of the data to date (June 2010), I was left scratching my head.  With the exception of June and July of 2009, the rate has been relatively consistent.

Average certifications per month (2006-2010): 4,401

Average certifications per month (2010): 4,350

Highest certification rate (June 2009): 13,920

Lowest certification rate (July 2009): 689

In reality, the average number of people getting the PMP certification is down in 2010.

Have I judged PMI unfairly?  Is the quality of the certification the same as it ever was?

The paradox of PMP quality versus value

Ty Kiisel of @task wrote a commentary titled Is Certification Losing Value over at CIOZone.  The ongoing (recent) discussion has been about there being too many unqualified but "certified" people in the workforce, while there are also seasoned professionals who will not get certifications because they believe they no longer hold value.  Geoff Grane over at PapercutPM and I have been going back and forth on this for days now.  Ty said he didn't think we was ready to say the PMI's certification was irrelevant quite yet.  He added that he believed the burden of demonstrating the value of any certification rests within the certification body, in this case the PMI. I agree with him.

Unfortunately, I believe PMI is becoming a victim of its own marketing success.  Though there is an audit process in place, to verify if PMs really have the necessary education and experience to sit for the PMP exam, I don't believe there are enough auditors to do the job. They are overwhelmed by sheer numbers of applicants.  Let's not forget the PMI is a for-profit organization. They want more members.

This paradox speaks to potential quality, not value. I think the PMI marketing department has won the war with hiring managers, convincing them the credential has value. What the PMI needs to do now is sell the (global) project management community on the quality of its credential. Until that happens, you'll have some people racing to the nearest boot camp to get a certification, not to learn best practices.

Read the 47+ comments over at Geoff's blog

Read Ty's entire commentary over at CIOZone

Graphic source: Nature.com

Passing the Test of All Tests (PMI Audit)

He who pulls the sword from the stone shall be a PMP

So, you just completed all of the paperwork, detailing all of your applicable education and work experience.  PMI has given you the green light to schedule a date to take your PMP® exam.  But what if you are audited?  Wouldn't that just suck!?

Well, you met the requirements.  PMI didn't say anything so you must be in the clear.  Time to sign up to actually take the exam.  Through a link on the PMI website, you pick a testing center in your area and find a time that will work.  You enter your credit card number and click submit.  Surprise!  Instead of getting a confirmation page, saying thank you for paying for the exam, you get a "you've been audited" screen.  What the hell!?

So, as soon as they have your money, you go into audit limbo.  The audit process has started and there are only two ways out.  [1] You withdrawal your application to take the exam.  You will get "most" of your money back. [2] You complete your audit, submit your paperwork, and PMI allows you to take the exam.

With all of the figurative threats PMI publishes about not embellishing your experience, you better not have done it!  I would compare this to going to a job interview.  Your potential employer says they like what they see.  You can continue on in the interview process, as long as you take your resume back to each of the employers you've listed and get them to sign a document agreeing you actually did the work.

What if your application is audited?

PMI answers a group of questions about this. First things first, stay calm. If you didn't lie on your application, you have little to worry about.  You should have honestly detailed all of your work experience during the application process. For every project you submitted, you're going to be sent an audit report in PDF format. (see the 3 graphics to the left)

Page 1:  What you entered during the application process.

audit_page1

Page 2:  Stakeholder information and a line for them to sign, agreeing you did the work that you say you did.

audit_page2

Page 3:  PMI Address information.

audit_page3

The pain of original signatures

For every one of these packets, you're going to need 2 original signatures.  You can have a colleague, peer, client or sponsor who has intimate knowledge of the project sign the audit.  You will then put the signed audit into an envelope (addressed to PMI), seal it, and have the same person sign the sealed envelope seam.  Once you get all of your packets signed, sealed, and signed, you mail all of the envelopes to PMI.

The actual audit by PMI will only take about a week. You will receive an email notifying you if you pass or fail.

What is my advise?

Go into the application process with the expectation of being audited.  Identify the people you want signatures from while doing your application.  Identify a backup (signature) for each project.  I spent more time tracking people down and getting signatures then I did completing the original application.  I had 20 signatures I had to get.  When budgeting the cost of the PMP exam, don't forget the cost of dinners and drinks for people who you need to track down to sign your audits.  I had to manage my audit process like I would a mini-project.

Is it all worth it?

From my personal and professional experience? Yes. Though I do believe the ever-increasing number of PMPs in the market may commoditize the certification, it's still in high demand. Regardless if you're a good PM, if you have a PMP® or CAPM®, there is an assumption you're a good Project Manager.  Because I was audited, I feel everyone should suffer the same scrutiny.  Perhaps there will be fewer paper PMPs, if they knew up front they would be audited.

Thank you to Joseph Gruber for verifying the current audit process.  He also suffered through a PMI audit.  I remember the PMI audit process being a little different, when I went though it back in 2006.  His personal experience was very helpful.

Did I miss anything specific you wanted to know?  Just leave a comment.

(Yes, that's a picture of me pulling the sword from the stone)

Meeting PMP Eligibility Requirements

When I was completing my PMP application, back in 2006, I recall reading the eligibility requirements and asking myself where I had the greatest gaps in my project management experience.  PMI did a good job of listing the process groups and activity "buckets" in which I could associate my time.  To visualize my strengths and weaknesses, I identified each activity provided by PMI as a process group subcategory and then associated project hours within a spreadsheet.  Though PMI had a requirement that I document experience in each of the process groups,  I had a personal requirement that I improve where my skills were lacking. This post isn't about my strengths or weaknesses, though you could assume by the graph that it would be Initiating and Closing.  It is about my identifying my experience gaps and helping you identify yours (in the eyes of PMI).  If you're a PMP or an aspiring PMP, take a look at the attached.

Step 1: Review the Process Group Activities PDF.  It will define the subcategories.

Process Group Activities

Step 2: Associate subcategory hours on a project basis.  The formulas are already in the worksheet.  All you need to do is add your hours to the Project Data sheet.

Activity Breakdown By Process Group

Step 3: Review the Graph on the tab titled "Graph".  If you don't identify your strengths and weaknesses from the data sheet, you will certainly see them in the graph.

Best Regards,

Derek