d0f8616e11
commit 279428df888319bf68f2686934897301a250bb84 upstream.
The scale value for the temperature channel is (assuming Vref=2.5 and
the datasheet):
376.7897513
When calculating both val and val2 for the temperature scale we
use (3767897513/25) and multiply it by Vref (here I assume 2500mV) to
obtain:
2500 * (3767897513/25) ==> 376789751300
Finally we divide with remainder by 10^9 to get:
val = 376
val2 = 789751300
However, we return IIO_VAL_INT_PLUS_MICRO (should have been NANO) as
the scale type. So when converting the raw temperature value to the
'processed' temperature value we will get (assuming raw=810,
offset=-753):
processed = (raw + offset) * scale_val
= (810 + -753) * 376
= 21432
processed += div((raw + offset) * scale_val2, 10^6)
+= div((810 + -753) * 789751300, 10^6)
+= 45015
==> 66447
==> 66.4 Celcius
instead of the expected 21.5 Celsius.
Fix this issue by changing IIO_VAL_INT_PLUS_MICRO to
IIO_VAL_INT_PLUS_NANO.
Fixes:
|
||
|---|---|---|
| .. | ||
| Kconfig | ||
| Makefile | ||
| ad5064.c | ||
| ad5360.c | ||
| ad5380.c | ||
| ad5421.c | ||
| ad5446.c | ||
| ad5449.c | ||
| ad5504.c | ||
| ad5592r-base.c | ||
| ad5592r-base.h | ||
| ad5592r.c | ||
| ad5593r.c | ||
| ad5624r.h | ||
| ad5624r_spi.c | ||
| ad5686-spi.c | ||
| ad5686.c | ||
| ad5686.h | ||
| ad5696-i2c.c | ||
| ad5755.c | ||
| ad5758.c | ||
| ad5761.c | ||
| ad5764.c | ||
| ad5766.c | ||
| ad5770r.c | ||
| ad5791.c | ||
| ad7303.c | ||
| ad8801.c | ||
| cio-dac.c | ||
| dpot-dac.c | ||
| ds4424.c | ||
| lpc18xx_dac.c | ||
| ltc1660.c | ||
| ltc2632.c | ||
| m62332.c | ||
| max517.c | ||
| max5821.c | ||
| mcp4725.c | ||
| mcp4922.c | ||
| stm32-dac-core.c | ||
| stm32-dac-core.h | ||
| stm32-dac.c | ||
| ti-dac082s085.c | ||
| ti-dac5571.c | ||
| ti-dac7311.c | ||
| ti-dac7612.c | ||
| vf610_dac.c | ||