FAQ - Benchmark scoring method (the stars)

We have attempted to rate the lights we have tested with a 5 star rating, in a relatively subjective manner, based solely on the specifications of the light. This reprsents our opinion on what is important from a technical point of view.

 

Step 1: Give points to each light based on certain metrics (see below).

Step 2: Total up the points to get an overall score.

Step 3: Take the range of scores across all the lights tested and divide the range into 8 ranges.

Step 4: Assign a star value (1 to 5) to each light depending on which range it falls into, including half stars.

 

Metric	Description	Formula
Lumens	Total light output of the globe. At the moment LEDs lights are often dull but rarely too bright. So we consider more Lumens to be a good thing. The denominator of 50 was chosen somewhat arbitrarily. A standard 60W incandescent globe outputs around 800 lumens, which would give 16 points.	Lumens / 50
Watts	Electrical power used. Which also directly relates to the ongoing running costs. So less Watts is a good thing. At least until very low wattage are reached. A drop from 50W to 25W is a substantial gain, but a drop from 3W to 2W is of much less interest.	log(100/(max(Watts,3),10) * 10
Purchase price	Everyone likes a bargain.	5 * log (100 / Price)
Estimated life	The longer the life time of a bulb the better. We don't give this too much weight becuase we don't believe all the claimed figures. Plus we don't think the difference between 10, 20 and 30 years life is significant. In 15 years LED lights will be far better and you'll want to replace them even if the old ones still work.	min (Lifetime Hours, 50000) / 5000
CRI	The color rendering index. It is the ability of a light to illuminate objects and make the colors look natural. Higher values are generally better, but the CRI measurement is somewhat controversial and not accurate in all circumstances. We assume values below 60 (or having no specified value available) are equally bad.	(max (CRI,60) - 60) / 5
PF	The power factor. A high power factor is a good thing. Power factor values are always between 0.0 and 1.0.	PF * 5
Running Temperature	The running temperature of the globe at 25°C ambient air temperature. Low temperatures will allow the globe to last longer and also produce higher output after their warm up period. In our opinion there is no great benefit to going below 50°C.	90 - (min(max(50,Temp),90)) / 4
Dimmable	How well the light dims compared to hypothetical perfect linear dimming. No points are given for globes that don't dim at all.	Dim rating / 2
Warranty period	How long the warranty period is. For LED lights which appear to have no stated warranty we assume they have 6 months. We cap this a 7 years, as we are dubious that any company would honnor warranties that far in the future.	min(max(Years,0.5),7) * 2
Flicker	How stable the light output is under nominal conditions. Measured with a compatible transformer, if required, but without a dimmer. Lower values for Flicker Percent and Flicker Index are better. Typical values are 0% to 100% for Flicker Percent and 0.0 to 0.5 for flicker index.	6 - ((FlickerPercent + (FlickerIndex * 200) ) / 25
Difference in claimed and measured performance	How misleading the claimed wattage and lumens are compared to their measured performance. Points are deducted if the total percentage difference of claimed lumens to measured lumens and claimed wattage to measured wattage is more than 10%.	- floor((abs(claimedLumens-measuredLumens)/claimedLumens + abs(claimedWatts-measuredWatts)/claimedWatts)*10)*2
Reviewer manual adjustment	Normally not used. Will be used for exceptional cases where the light completely fails.	+-10

 

Example calculation

Here is an example for a LG Electronics LED globe.

Lumens: 9.64
Watts: 11.08
Price: 3.72
Hours (life): 6
CRI: 4.4
Power Factor: 5
Temperature: 5.2
Dimmable: 0
Warranty: 6
Flicker: 3.6
Overall Score: 54.69

 

Pros

• LED lights are complex (at least compared to incandescents). Most people don't have time to fully research and understand all the specifications of LED lighting. Using a star rating is a fast, familar, comprehensible way of providing a general indication as to the efficiency and quality of the light.
• The system isn't subjective in the sense that it is purely based on the collected specifications.
• The system will automatically cater for future more advanced lights, and yet to be discovered very poor lights. As new advances are made in technology and new lights are tested the scaling will automatically adjust.
• The system is automatic and consistent. Reducing our work load.
• The system is documented, reproducible and transparent.

Cons

• The relative weight given to each metric is just our opinion of what is important. Different people will have different requirements and different opinions. These people will find the star rating of limited applicability. For example the method is somewhat weighted towards lights with high efficiency and high output. Cost is a slightly less important factor in the score.
• The star rating aren't totally stable over time. As new (better) lights are tested, the old lights will get ranked lower. This might be a Pro or Con depending on your point of view.
• Factors which might be critically important to some people like beam angle, a manufacturer's reputation and the physical appearance of the globe are not considered at all.
• Fancy or unusual features, like remote controls, adjustable color, suitability for outdoor use are not considered in the score.
• All types of lights are ranked using the same method. But downlights as a rule aren't as efficient as the globes. The consequence of this that there are no 5 star GU10 downlights.

 

Related Pages

• Can I send you LED lights for testing?
• How accurate are our systems of measurement?
• What is the life expectancy of LEDs?

 

Back to FAQ index