An organoleptic taste panel assessment means a team of 10 people have taste-tested the olive oil.

For an olive oil to be labeled extra virgin, a taste panel just has to find no taste or smell defects with the olive oil.  For an olive oil to be labeled ultra premium, they have to meet a certain criteria for 3 categories: Fruitiness, Bitterness, and Pungency.  (Minimum 3.5 for fruitiness, minimum 2.0 for bitterness, and minimum 2.0 for pungency)

The taste panel assessment also makes it very easy to help people figure out which olive oil they like – if you find yourself enjoying an olive oil that has a high bitterness score, you will generally like other olive oils that have a similar score.