Wow, that’s the worst I’ve heard, my discrepancy averages a tad under 2°F.
My thermapen reads between -0.4C & -1C of the Anova (tested up to 65c so far). Considering it’s a few years old and was only calibrated at the factory, I think that’s fairly acceptable. Haven’t seen any wild inaccuracies.
I just tested my two-week old model using the readout on the Johnson Controls A419 Digital Thermostat Control Unit that we use for sous vide cooking with our crock pot before we got the Anova product, and an iGrill electronic digital meat thermometer that connects with my iPhone by bluetooth. The readings on the two monitors consistently show the Avona to be somewhat higher than the temps they both display, testing with the Avona set at 170º, 160º, 150º, and 140º (2º high); 1ºF high at 130º, 120ª, and only 0.3ºF high when set at 110º. I’m guessing that the Avona iPhone app calibration will handle this near-linear error with aplomb.
I’ve made a little plot of my measurements using my brand-new Anova versus my brand-new Thermapen. You can see the results here (scroll to the bottom for the graph). The uncertainty is not linear, but it is very nicely affine-linear. I had to use the app to adjust the offset by -5.5F (which seems huge to me), but once I had done that, the Anova was within the uncertainty of my Thermapen across the useful range of temps. If I want to be particular, for temps above 150F, I’ll have to bump up my Anova set point by one notch to get agreement with my Thermapen.
All in all, it doesn’t give me a great deal of confidence in this “precision” circulator. Users shouldn’t have to fork over another $100 for an accurate thermometer after forking over all the money for the Anova. I’m not expecting Thermapen-like accuracy, but 5.5F is far worse accuracy than my two old rusty cheapo meat thermometers…
@MOB Got it, a temp offset of 5+ can sometimes linked to an earlier device but that’s obviously not the case here.
There’s also the chance of some amount of inaccuracy with the Thermapen. Putting two Thermapens in the water against each other will often give different temp reads. The Thermapen might read, for example, three degrees above temp and the Precision Cooker may read two degrees below temp, so it’ll make it seem like the Precision Cooker seem off by more than it actually is. For good measure, the Precision Cooker devices are measured for accuracy against multiple commercial kitchen and lab thermometers.
I measured both an ice bath and boiling water with the Thermapen; it was within 0.3F of the expected result for both. Both my meat thermometers are much closer to it than the Anova… for whatever that’s worth. My first attempt with the Anova resulted in raw steaks – consistent with an error in the region of 5-10F. Now that I’m going by the Thermapen, meat and eggs are coming out beautifully every time.
The Thermapen is brand new, by the way. Thermoworks also calibrates their devices, but – at least with new Thermapens – they have a very good reputation, and clearly this new Anova device has had some quality-control challenges. So basically, I trust the Thermapen to within its stated accuracy, whereas I don’t trust the Anova. This particularly worries me because I’m cooking for a pregnant friend, and I’m really afraid of serving her underdone meat. (That was actually my original motivation for buying the Anova, and is why I broke down and bought a Thermapen last week when it became clear that the Anova was wrong.)
Is it possible I was sent an old device accidentally? The tube it came in was scratched up even though the box the tube came in was fine. And there were (small but plainly visible) scratches on the screen when it arrived.
@MOB Totally, Thermapens are much more accurate than meat thermometers and it’s pretty cool that you did all of this testing and shared the results. You’ve done a thorough job and I don’t doubt the unit is the issue here - I hope it didn’t seem like I was skeptical, and I’m so sorry if it came off that way! Just wanted to give a little detail around the testing the devices go through at the lab for background. If you wouldn’t mind, I’d like to share your results with the scientific team as they’d be interested in your experiment and will probably want to look into it to identify the issue.
I don’t think it would be an earlier unit but it’s always possible - is the screen protector on perhaps? It makes the screen look scratched and there isn’t a tab on it to indicate that it’s there so it’s very difficult to even tell that the device has one.
Have you done the re-calibration for the device according to the Thermapen’s temp read using the app?
If you wouldn't mind, I'd like to share your results with the scientific team as they'd be interested in your experiment and will probably want to look into it to identify the issue.
Sure, that sounds great. I’d be happy to discuss it, or take additional measurements if they think of anything useful I could try.
I don't think it would be an earlier unit but it's always possible - is the screen protector on perhaps? It makes the screen look scratched and there isn't a tab on it to indicate that it's there so it's very difficult to even tell that the device has one.
I don’t think so. I never saw a screen protector on it, but if there’s one on there now, it’s stuck pretty tight…
Have you done the re-calibration for the device according to the Thermapen's temp read using the app?
Yes. I adjusted the Anova set temp until the water was about 135F (because that’s about the middle of good meat temperatures), waited for everything to settle down, then entered the difference between the Anova-set and Thermapen-measured temperatures in the app. That’s the -5.5F I mentioned above. So they now agree perfectly right near 135F – though there are small errors away from that temp.
@MOB Great, thank you! I’ll send it over to them.
The unit should have had a screen protector…hmm…it may have been removed at testing but that’s not likely.
Could you send over an email with a photo of the back serial code? I’ll PM you with my email address.
Thank you!!
The unit should have had a screen protector...hmm...it may have been removed at testing but that's not likely.
Oh yeah, you’re right. I guess my tiny fingernails just weren’t up to the challenge. My girlfriend got it off no problem. The screen protector was very neatly applied. Looks much better now, though. 
@MOB Haha, perfect, thanks for the update. Yea…we’ve got screen protector pull tabs on our to-do list. Needless to say, this happens a lot!
Your thermapens may be nonlinear. Cypress semiconductor has excellent app notes on how to linearize negative temp thermistors. These are not easy to linearize being inherently nonlinear and are usually not laboratory accurate, but inexpensive for commercial application.
Do you have access to a thermocouple meter for reference. Some of the stores now sell multimeters with type k thermocouples inexpensively. You can check temps against triple point ice and water temp, and water altitude corrected boiling point. Or just enjoy the sous vide unit, check against a series of eggs at different temps. If its way off send it back, Anova's cal against NST standards is better than home. Better yet, order a second unit and compare the two. Then you still have one if you need to send the other back. OR Then you have two for multiple temps :) simultaneous cooking or twice as much for guests over. Whats better than one Anova. Another for portable use.
Does anyone know if the ANOVA probes are thermistor (most likely) or thermocouple, not that it makes any difference in use or enjoyment?