This post is not specifically about FAR (Federal Acquisition Regulations). It's about using relative grading or estimates (evaluations), before absolute grading or estimates (evaluations). As an advisor to a source selection committee, I'm not allowed to write or talk about what is happening behind closed doors. I can, however, write about the FAR . Don't click away just yet!What exactly does the FAR say?
Subpart 15.303 Source Selection Responsibilities
Paragraph (b4) The source selection authority shall ensure that proposals are evaluated based solely on the factors and subfactors contained in the solicitation (10 U.S.C. 2305(b)(1) and 41 U.S.C. 253b(d)(3)).
That means is if you have 10 proposals, you are not allowed to compare them. Based on content contained within them, you can not say one proposal is better or worse than another. You may only grade the proposal against the solicitation, as if it was the sole submission. So, this is where I see the problem with the FAR. After reading Predictably Irrational, by Dan Ariely, I know humans don't think like this. We compare things and then react (usually irrationally). We're also pretty bad at estimating if we don't have something to use an a baseline or anchor.
The only proposal submission that will truly comply with the FAR is the first one read. The rest of the proposals will be compared to all those read before it. This isn't intentional. It will happen subconsciously. Though the FAR is attempting to be fair, I don't think that it's realistic. If this was a scientific experiment, you would re-baseline after each test. But the human memory is a tricky thing. You can't just reboot and clear the cache.
I've seen the same thing happen with providing task estimates. You can spend a lot of time and money trying to get an absolute estimate. For some projects, they require that up front. But, where there is a lot of uncertainty, relative estimating is commonly going to get you closer to where you want to be. You many not be able to accurately predict how long it will take to complete a task but you can say if the task being estimated will be easier or harder than a similar task recently completed. Use that as the anchor.
What I am proposing with the FAR and with estimating is provide a relative evaluation (bigger/smaller, better/worse, sm/med/lg...) based on all available information. As more information becomes available, refine your evaluation or estimate to absolute terms (SLA1=4, Task1=Nhr(s), Task2=NDay(s)...).
If I were to change the FAR, I would require all of the proposals be reviewed once (establishing an overall baseline) and then focus my attention on the best proposals. But, since I'll only be advising the source selection committee for two more days, I'll leave it up to the FAR.
Which do you prefer, relative or absolute estimates?
Yes, the book link is an affiliate link.