|
| Iniciado por guest, 17,nov. 2014 11:27 - 7 respuestas |
| |
| | | |
|
| |
| Publicado el 17,noviembre 2014 - 11:27 |
Hi guys,
Has anyone used FlowHeater (database Import/Export) with HF (via ODBC)?
<a class="ExternalLink" rel="nofollow" target="_blank" href="https://www.flowheater.net/">https://www.flowheater.net/</a> |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 14:47 |
...reason for asking:
I have a requirement where there will be 100's of potential suppliers brought online at various points and each supplier will provide Excel, CSV with the possibility of more formats of file over time. The files will be supplied on a regular basis all with (potentially) their own layouts. Main use will be product inventories.
It seems that FlowHeater fills this void between my system's inventory layout and the data being supplied thus avoiding me to having to write individual import procedures for each supplier and also avoids having to redistribute packages or supply system patches as more import files come online.
Do you guys have a similar requirement and if so how are you solving it? |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 16:42 |
Hi Darren,
What we did is a simple screen where it is possible to "map" a CSV / XLS file columns (header) to the corresponding field in our database. In so doing we are able to import all data fields our system supports.
In another product we have, this "simple screen" is much more complex. One can map any "File" in the analysis with any external file. You can import and export.
It's quite easy to do but needs some testings with all kinds of data. And from there you could simply save patterns and have it work automatically: like Save these settings for "supplier A". All mapping is done and tested only once. Then afterward, everything can be automated completely.
Best regards,
Alexandre Leclerc |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 16:58 |
Hi Alexandre,
Thanks for getting involved... :spos:
Yes, I too starting down a route similar to your more complex solution but it soon started to get more and more complex as new functionality started to come into play, things like; setting fixed values like date and times, setting values based on calculations and scenarios like importing a product file and the need to recognise that a product already exists and should be updated and not created - which is similar to the LOOKUP feature in FlowHeater. |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 20:17 |
Hi Darren,
Indeed, you must add this little features. This is what we did. In the end, once it is done, it just works. We did as you did : add if does not exist, update if exist. For personalized scripts, it's quite easy too without code change: we use a "script" file with WLanguage in it. So we simply code standard procedures. But it would cost you much more that buying this software you're looking into because the price is very low.
But I guess the solution you are looking is doing that in some way. It will require setup time once then you live with it. Only the testing and integration could take you some more time, and playing around with HFSQL ODBC drivers. Testing their trial will give you an idea (but take some of your time - part of the "cost").
Maybe your project is too complex and big. But if this is only for the inventory, my guess is that 1 or maybe 2 days of work would be enough to have a flexible interface for your customers with the required specs. (This is about the time it took us to build these interfaces in our software.) Add one or two days if you want to go into the scripting thing.
Best regards,
Alexandre Leclerc |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 20:29 |
Hi Darren
We had a similar issue & use a mapping concept but expanded somewhat. We have created single generic import/export classes and store all the information in a memo field in a data table.
This field looks something like
Code{3} TaxType|S|IF {5} ="A" THEN DataValue="R" ELSE DataValue = "G" Dept|F|"ABC" Price|P|TEN_PriceProc({6},{8})
Anything in curly brackets is replaced by ExtractString where the number is the field in the import file. The items separated by | | tell the class to take special action where S is a simple procedure (basically a 1 liner), P is a complex procedure the details of which are stored in a separate table and which can return multiple values, F is a fixed value and so on. Our classes create the dynamic procedures (only once for the entire import file) and it all works pretty quickly.
We have stipulated that files must be either comma or tab delimited but this could easily be expanded if required. This whole thing took a fair bit of work but it gives us a great deal of flexibility. If a new definition is required we can simply send the user a text file to import into our table and they're away. If they have any technical knowledge they can even write their own if they wish.
So far we have 6 different import/export definitions using this method successfully. One other thing we have done to minimise import processing is store a CRC value against each record. The first time a file is imported that gets written for each record but for subsequent imports we calculate the CRC on the import record; if it matches the stored one we skip that. This has made an enormous difference on processing time when the whole inventory table is supplied every time.
Cheers
David |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,noviembre 2014 - 22:26 |
Hi Darren
With the kind of requirement you are giving us, I would definitely NOT use a third party tool, as anything not available in it that you need for one import would be completely blocking.
Now, to complete Alexandre's answer... Once you have the basic matching and saving of the pattern, you may want to include a dynamic compilation feature... This way, you can massage the data before importing it.
In the mots comprehensive example of that function that I had to code, we had compiled script possible:
- before importing the file (convert from one format to another, by example)
- after importing the file (move into another directory for another processing, allowing to chain processes, by example)
- before importing a record
- after importing a record
- For each field
This allowed us to record any specific controls, file transformation, field conversion, etc, etc, as wlanguage script and compile them run them when importing...
Once built, it was very easy to use, especially as each scrupt was reusable, copyable, etc.
Best regards |
| |
| |
| | | |
|
| | |
| |
| Publicado el 18,noviembre 2014 - 12:10 |
Hi guys,
Thanks for getting involved - it's always appreciated...
Lots of food for thought.
I agree, it's probably best not include 3rd party s/w into the mix, especially as I'm quite a way into developing my own solution along the lines of the discussions above, hence starting this thread, but it's always good to have an open discussion and get confirmation <img src="/NG2013_WEB/ui/smiley/1.gif" align=absmiddle border=0 alt=":)">
I'm going to carry with my bespoke solution on but bearing the above comments in mind...
Thanks again! :cheers: |
| |
| |
| | | |
|
| | | | |
| | |
|