is to see how well the interviewee fits in with the group. Everything beyond that is gravy. Someone with all the qualifications in the world isn't going to be any good if they don't fit in with the group, and someone who fits in well but may lack some qualifications (within reason, of course) can be taught -- and they'd need to be taught the specifics for that organization anyway.

@Joe - yes I suppose if they did not collect rating immediately after each candidate interview, everyone would get totally lost.

@jabailo - you reminded me of the time, when I was asked to sit at a computer and given actual problems and I had to provide written solutions within a set time frame.  

Many companies now shift many of the boilerplate issues to the phone interview, saving valuable time in the overall vetting process.   It seems to me when 18 people gather to interview a candidate, it's mostly posturing for their own status and  position and the interviewee becomes a pawn in an institutional game.

If the issue is information...about the candidate, you'd probably get better answers using the phone and social media.  Put up a question, let the candidate think and then respond.   Only at the final stage would you bring the person in (and by then it should only be about 3 candidates) and then make a final decision.

I think a lot of times candidates should almost be paid for their time interviewing, as employers can use them for information pumping and other activities not related to them getting the job.

Interviewing the 18 people that way screams to me that they had no idea what they were doing in terms of interviewing, and had no idea what they wanted.  Indeed, most of the questions they threw together are cliche -- which is probably why they had trouble deciding and fell back on the unimaginative metric of largest past budgets: because the questions are designed in such a way that it is unlikely that the candidates would give answers that were significantly better or significantly worse than any other candidate.

Thanks everyone for input and questions.  Trying to respond to all.

"Best candidate" is often the best fit in a particular position in a particular organization and not necessarily the strongest in the pool.  Questions selected should allow candidate to respond to the needs of the organization. I thought this set of questions were great.  Fit would be easier to determine since answers would reveal thought process, approach and overall mind-set.  It would also allow candidates to demonstrate ability to communicate effectively within time constraints.

The problem was using 18 people to make a very subjective judgement of fit.  The prescreening process had already determined that all candidates invited to campus were well qualified - surely they could all do the job -- the key now was best fit.  If you were going to use 18 people then a decision-making process should have been pre-established and everyone should have been forced to rate every candidate on predetermined criteria which were important to the institution.  

If budget size was important, then it should have been used before the interviews and not after.  The value of this excellent set of questions was wasted.  Scheduling 18 people for 10 candidates must have been a nightmare as well.  

Yes I have also experienced and enjoyed campus interviews where you had to present to an open audience of students, faculty and staff.  Feedback is collected but decision-making is done by a much smaller set of people.

It seems to be increasingly the case with interviews that questions are pulled from standardised sources.  What do people really base judgments on?  Whether they feel comfortable with you?  Whether the resume checks out?

It usually goes something like this...we need representation from administration, faculty, unions, support staff, IT direct reports, IT staff, peers, women, minorities and so on = 18.

This approach is often to diffuse the accountability. So if the person doesn't work out, they can say they were all fooled and no one gets blamed.

The questions are often pulled from a set of standard Q's for all hiring with some specific to the job. However, they rarely address the things that matter in performance (re: things that get you fired) and stick to non-specific things.

I was once interviewed for a job by an entire seminar of students.  Their professor was doing the hiring, but he let them grill me.  It was actually fun - but of course a lot less pressured than the kind of panel Mansur describes.

What was it that you would have changed? I am curious because of the title? 

I wonder how the college decided to involve so many people in the interview process. I'd guess it snowballed until it was out of control, with each department head asking to be part of the interviews since IT works with everyone. This approach is, as you describe, extremely inefficient, especially when it comes down to actually selecting a few finalists. There is no way for so many to agree on two or three prospects. 

If the college wanted to limit candidates to those responsible in the past for a specific budget amount, they could have done so in the application process, saving those executives without this experience the time of coming to this mass interview.