Step 1: IPCC Approves Outline
Government representatives and full-time scientist members of the panel first write and approve a rough idea of all the topics to be covered in the specific report to be published. Ostensibly this step is only there so that the report can have a theme, but whenever you write an outline in advance of the data you're researching, you have already committed to a series of assumptions about the data you're going to find. Imagine if I were doing a double blind study of a new cancer treatment and I'd written the outline of my findings prior to even doing the testing? Then I proceed to step two... the cherry-picking stage.
Step 2: Governments, Organizations Nominate Experts
Now that we know what we want to say this is the part where the government officials on the IPCC get to decide which scientists they will be using to make their case. Also... a great opportunity for bias to influence the overall results. We do this all the time as ordinary individuals of course, and I'm fine with that in general, but I don't have the power to force someone else to agree with me whereas the IPCC ultimately will in many ways. So... good times here as well.
Step 3: Bureaucrats pick the specific authors
So - NOW that we've already chosen the group of scientists we'd like to use to support the outline we already wrote for the report, now we should choose the specific scientists we'd like to have author the report itself.
Does anyone else not see a bit of a pattern here? Government officials - non-scientists - have an amazingly large amount of control over the way this report comes out simply by being in control of the level of diversity of opinion represented.
So... Skipping ahead a bit, we find that after one round of purely expert (peer) review - we get a 2nd Draft written by the selected authors...
Step 7: Government & Experts Review Draft
Out of curiosity - why exactly would the governments represented by the IPCC need to verify a scientific report?
Perhaps they need to make sure it conforms to their original outline... Just a thought.
Now, this goes through a few other government review steps, and that's all well and good, but then the government officials on the panel get to write the Policy Summary...
This is what (as I'm sure many have noticed) really chaps my ass. Because at this point, the politicians can monolithically make sweeping recommendations about what new laws to write, new programs to invent and new tariffs & taxes to pay for it.
So, just to review:
1. The policy makers write an outline.
2. The policy makers nominate scientists to write a report based on said outline.
3. The policy makers pick the specific scientists from their pre-selected pool to write the actual text of the sections of the report again based on said outline.
4-5. Draft 1, peer-reviewed (GOOD!)
6-7. Draft 2, government (and peer) reviewed... what!?
8-9. Final draft, government reviewed... what1?X10
10. Panel approves the report and publishes to the world.
And bonus step 11: IPCC gets to make authoritative recommendations of policy.
Anyone ever heard the term "selection bias"? For some strange reason, it keeps popping up in my mind.
Now, to be clear, I don't think the scientists are purposefully doing bad research! I think people who need backing for policy positions are put in a very easy position to generate reports and authority through selecting the people who are likely to represent their views. And just to be sure, they get to write the outline in advance.
Where did I get all this you might ask!?
Oh yes, the IPCC website itself.
http://www.ipcc.ch/ipccreports/index.htm
Government representatives and full-time scientist members of the panel first write and approve a rough idea of all the topics to be covered in the specific report to be published. Ostensibly this step is only there so that the report can have a theme, but whenever you write an outline in advance of the data you're researching, you have already committed to a series of assumptions about the data you're going to find. Imagine if I were doing a double blind study of a new cancer treatment and I'd written the outline of my findings prior to even doing the testing? Then I proceed to step two... the cherry-picking stage.
Step 2: Governments, Organizations Nominate Experts
Now that we know what we want to say this is the part where the government officials on the IPCC get to decide which scientists they will be using to make their case. Also... a great opportunity for bias to influence the overall results. We do this all the time as ordinary individuals of course, and I'm fine with that in general, but I don't have the power to force someone else to agree with me whereas the IPCC ultimately will in many ways. So... good times here as well.
Step 3: Bureaucrats pick the specific authors
So - NOW that we've already chosen the group of scientists we'd like to use to support the outline we already wrote for the report, now we should choose the specific scientists we'd like to have author the report itself.
Does anyone else not see a bit of a pattern here? Government officials - non-scientists - have an amazingly large amount of control over the way this report comes out simply by being in control of the level of diversity of opinion represented.
So... Skipping ahead a bit, we find that after one round of purely expert (peer) review - we get a 2nd Draft written by the selected authors...
Step 7: Government & Experts Review Draft
Out of curiosity - why exactly would the governments represented by the IPCC need to verify a scientific report?
Perhaps they need to make sure it conforms to their original outline... Just a thought.
Now, this goes through a few other government review steps, and that's all well and good, but then the government officials on the panel get to write the Policy Summary...
This is what (as I'm sure many have noticed) really chaps my ass. Because at this point, the politicians can monolithically make sweeping recommendations about what new laws to write, new programs to invent and new tariffs & taxes to pay for it.
So, just to review:
1. The policy makers write an outline.
2. The policy makers nominate scientists to write a report based on said outline.
3. The policy makers pick the specific scientists from their pre-selected pool to write the actual text of the sections of the report again based on said outline.
4-5. Draft 1, peer-reviewed (GOOD!)
6-7. Draft 2, government (and peer) reviewed... what!?
8-9. Final draft, government reviewed... what1?X10
10. Panel approves the report and publishes to the world.
And bonus step 11: IPCC gets to make authoritative recommendations of policy.
Anyone ever heard the term "selection bias"? For some strange reason, it keeps popping up in my mind.
Now, to be clear, I don't think the scientists are purposefully doing bad research! I think people who need backing for policy positions are put in a very easy position to generate reports and authority through selecting the people who are likely to represent their views. And just to be sure, they get to write the outline in advance.
Where did I get all this you might ask!?
Oh yes, the IPCC website itself.
http://www.ipcc.ch/ipccrep
No comments:
Post a Comment