[FR] Price parsing in data importer should allow more formats

[hvraven]

Please verify that this feature request has NOT been suggested before.

• I checked and didn't find a similar feature request

Problem statement

Currently the data importer expects price information to be a valid float value. When using the importer to upload data for a purchase order this easily fails, as vendors format their data differently.

The concrete example I ran into is a shop, which adds the currency to the price, i.e. the entries are 1.23 €. Additionally there is the extra annoyance of the German tendency to use , for separation, so we also have a mixture of 1.23 and 1,23 (plus 1,23 €).

Suggested solution

As the importer knows based on the column, that the data should contain prices, it should drop any added currency symbols on front or back (any non-numbers).
Ideally it would also detect "german" notation (maybe others use it as well), and translate it to proper floats before parsing.

Describe alternatives you've considered

None

Examples of other systems

No response

Do you want to develop this?

• I want to develop this. (I need some pointers where to start. I know a lot of Python, but barely any TypeScript and don't know there such parsing should be located)

[matmair]

Would certainly be a good addition. All code for imports is in src/backend/InvenTree/importer - I do not think there will be a need for a lot or even any typescript

[SchrodingersGat]

We also handle money values using custom classes already - so potentially these could be adjusted to provide automatic conversion when de-serializing from an imported file:

InvenTree/src/backend/InvenTree/InvenTree/serializers.py

Line 35 in e0559bb

class InvenTreeMoneySerializer(MoneyField):

InvenTree/src/backend/InvenTree/InvenTree/fields.py

Line 80 in e0559bb

class InvenTreeModelMoneyField(ModelMoneyField):

[SchrodingersGat]

@hvraven any thoughts on this? Happy to help where I can

[hvraven]

I started setting up a dev environment and dug a bit in the code spots mentioned. I would hook the logic into the extract_data function. It already handles parsing of booleans and dates as special types.

InvenTree/src/backend/InvenTree/importer/models.py

Lines 622 to 625 in e0559bb

	if field_type == 'boolean':
	value = InvenTree.helpers.str2bool(value)
	elif field_type == 'date':
	value = value or None

Currently the currency entries have the field_type set to decimal. Could that be changed to a special currency type? It would be a decimal, but with the special meaning of encoding currency. Or would that break a lot of other stuff. I couldn't immediately find the definition where the field_type is defined.

[SchrodingersGat]

Could that be changed to a special currency type?

We could probably support this. Have a look at how the custom serializer classes I linked above work

[SchrodingersGat]

@hvraven looking into this further, we could update the InvenTreeMoneySerializer class to provide more "lenient" parsing of pricing information.

However - we would need to write a parser that can handle different locale formats, etc. Currently the only alternative solution is to properly sanitize the dataset before importing.

Would you like to look into this? Otherwise I don't think it will receive any priority from the core dev team