Opened 8 years ago

Last modified 8 years ago

#45 new defect

Calls to numpy.exp() result in under/overflow

Reported by: flip Owned by:
Priority: minor Milestone:
Component: other Version:
Keywords: Cc: bsoher

Description

Calls to numpy.exp() result in under/overflow

We've seen a number of places where calling numpy.exp() results in
something like this:

FloatingPointError: underflow encountered in exp

We have two approaches to solving this problem.

The first is to limit the data passed to exp() to some arbitary-ish value like so:

value = np.where(value >  BIGEXPO,  BIGEXPO, value)
value = np.where(value < -BIGEXPO, -BIGEXPO, value)
result = np.exp(value)

The other is to tell numpy to ignore these errors:

restore_these_settings = np.geterr()

temp_settings = restore_these_settings.copy()
temp_settings["over"] = "ignore"
temp_settings["under"] = "ignore"

np.seterr(**temp_settings)
value = np.exp(value)
np.seterr(**restore_these_settings)

In one case (in funct_svd_filter.py) we've seen this code create NaN values in the result which need to be dealt with.

The limit approach works OK for some data, but not others. For instance, there's a call to exp() in Analysis fitting when setting up the line shape. Cropping the data (eg. press cp0) to +/-60 is fine under normal circumstances, but things go badly if I change the option voigt/initial values/linewidth method to deconvolution. I get something like the attached file bad_fit.png. (plot a = raw & inital model, plot b = raw & weight array, plot c = raw and (fit + base), plot d = raw-fit-base.)

Another dataset that causes exp() headaches is is AC_W_raw_act in the philips_sdat_svs/human_phantom_stone in our sample data. That causes problems in the calls to exp() in _create_hlsvd_fids in funct_svd_filter.py.

Attachments (1)

bad_fit.png (137.0 KB) - added by flip 8 years ago.

Download all attachments as: .zip

Change History (3)

Changed 8 years ago by flip

comment:1 Changed 8 years ago by flip

I forgot to mention that one can find all places where we call exp() with this command (assuming one is on a system that has egrep):

egrep -r "(numpy|np).exp" *

It finds about 130 hits.

This command shows every place where we use numpy.seterr():

egrep -r "(numpy|np).seterr" *

comment:2 Changed 8 years ago by flip

The "bad fit" is actually my bad. I had an error in my safe_exp() implementation. Cropping data to +/-60 works fine in chain_voigt.py, contrary to my comment above.

Note: See TracTickets for help on using tickets.