PC SOFT

FORUMS PROFESSIONNELS
WINDEVWEBDEV et WINDEV Mobile

Accueil → WINDEV 2024 → Microservices
Microservices
Débuté par Art Bonds, 25 juin 2017 17:34 - 29 réponses
Posté le 25 juin 2017 - 17:34
What is the WD equivalent of the microservices architecture?

Would it be WDL's (the WD version of DLL's?), which are now discouraged? Components, which are included in a project... but doesn't seem to adhere to the microservices concept because they can not easily be replaced without compiling the whole program (or am I missing something about components?).

EDIT: On further thought maybe I should clarify. What I am thinking about is building a program made out of "microservices". To equate this to an automobile. We build version 1 of the automobile. As an improvement we want to change out the drum brakes with disc. Then swap out the carburetor with fuel injection. A new AM/FM/cassette for the AM/8 track, then further down the line of a AM/FM/Sirius CD player. In each upgrade, I only want to upgrade/code/replace that particular part of the car, not upgrade and have to compile the whole program.

Hope that made sense.
Membre enregistré
19 messages
Popularité : +2 (2 votes)
Posté le 26 juin 2017 - 08:52
If you have compiled the application to take patches you should be able to upgrad your existing users easily.






Though I have never used this feature personally till date.

--

Yogi Yang
Posté le 26 juin 2017 - 08:59
Hi Art,

My "KISS" approach to this is to deliver the whole system with all the features in a single exe, and then activate/deactivate certain features with a "secret code" that the user gets by email and enters into his main parameter file, where it sets the equivalent features on/off.

Then you will additionally only need a maintenance window for the code, where you select the customer from your own contacts file and set his features with checkmarks, and the window generates the "secret code". When you press the button to email to code to the customer, it will also be saved his contact record so that you can later retrieve it and change and re-send it, if the customer later finds out that he really wants to play his 8-track tapes:-))

Best regards
Ola
Membre enregistré
19 messages
Popularité : +2 (2 votes)
Posté le 26 juin 2017 - 10:17
On the second thought.

A developer called 'David Egan' has developed a module based solutions for creating modularized wd applications. Please approach him for a working code sample.

In his solution what you can do is build the apps functionality in separate modules and depending on the functionality required copy the modules in a predefined location to add the functionality/feature.

--

Yogi Yang
Posté le 26 juin 2017 - 13:43
Hello Art,

There is one big problem with your analogy. When you change the sound system on a car, it has NO bearing on the brake systems or anything else.

However, in your program, EVERYTHING will have a bearing on the ANALYSIS. So when yo change a subsytem, you will MOST OF THE TIME, also need to change the analysis and therefore all the other subsystems.

So, except for the rare cases where no file is shared between subsystem, I strongly disagree with your microservice approach.

I must also say that I've seen numerous people try that in the last 20 years, any and all possible ways (separates exe, wdls, components, etc), and in very case, they ran into the same problem: exchange of data between modules, and therefore analysis problem.

The only way you can achieve that is by having a separate analysis for each module, and do the data exchange via a webservice that is analysis neutral. So that means a LOT more work.

Personally, I put that concept in the category of the false good ideas...

Best regards
Posté le 26 juin 2017 - 16:52
Hi Art,

I'm with Fabrice on this one: Near to impossible with WX if you're depending on a WX analysis.
Without the use of the analysis, different WDL's would come the closest I guess but you'll loose schema versioning and many other advantages...

Other non WX technologies (OO only using an ORM (that has schema versioning built in like the .Net Core Entity Framework with its embedded 'migrations' concept)) will provide you a lot more possibilities in this area and than still... numerous constraints are likely to popup...

Just my 2 cents,

Peter Holemans
Posté le 26 juin 2017 - 21:18
Hi Ola,

While the hiding of everything in a large monolithic program and turning on/off specific functionality sounds good, what if there is some functionality that you did not think of or some feature that you didn't know of ahead of time? When you designed your ERP maybe the only music to be had was AM/FM/8 Track. Then someone invented cassettes... and there was no checkbox to activate the cassette functionality. Or the CD. Or Satellite radio. In each case you had to create/program something new, fold it into your already expansive and complicate ERP, and redistribute it to your thousands of clients. Then hope there wasn't a bug that you missed, and you get to do it again. That is what I was looking to guard against, having to add a detail and then have to recompile the whole kit and caboodle and redistribute just to get one thing updated.

Also, in my opinion, there are inherent drawbacks to using monolithic architecture... it quickly leads to monolithic hell for any large and/or complex application that needs to be modified or improved. I've already had to deal with a large monolithic program called WinDSS (a point of sale program still used here in the US). As retailers have found out WinDSS needed to be modified, sometimes heavily, to fit their business needs.

Adding any new functionality took an inordinate amount of time and was replete with unanticipated bugs and side effects. Cross application changes were a nightmare, and supposedly simple changes affected other parts of the program in ways unanticipated and frequently undetected until it hit the stores.

The complexity buries you quickly once the application grows from it's simple roots to the enterprise level. Integration of applications are difficult except through the database, which I think was Fabrices point.

Over time the code became messier and messier as more maintainers worked with the same code, putting patch over patch until entire sections need to be refactored or completely rewritten (been there, done that).

I have always said WinDSS was like a balloon... push on one area and it bulges somewhere else. But as a WinDSS developer I can't complain to much. WinDSS had made me millions at the expense of multiple retailers.

In a program written with a microservices mindset, each application (like CRM) is deployed independently of other applications (like Product Management, or Shipping and Receiving, or Pricing, or Data Management). Each application can be developed, improved and deployed independently, making it easier to deploy new versions of applications frequently. It is (supposedly) easier to scale development. A completed program has improved fault isolation. If there is a error in one application, the other applications will continue to handle requests and do their thing. In comparison, one misbehaving component of a monolithic architecture can bring down the entire system. And when the fix comes only a small part of the program has to be replaced.

I've experienced monolithic already, I was just hoping to find a different way, and the microservices model seemed a good fit. But Fabrice and Peter may be right, I will have to figure out and post a response about my ideas regarding their Analysis issues.
Posté le 26 juin 2017 - 22:27
We use a main program that is the app.exe including login and menu of all the available modules (= dll, that in wx is a wdk component). The exe does also the DB connection.

Then the main app calls each wdk when is needed. The only "heavy" condition is having ONE analysis shared beetween the main app project and all wdk projects.
Even if you define a local analysis in wdk projects, at runtime they will use exe's analysis. If a datafile is only local, the app will crash, you have to insert it also in exe's analysis.

This is because all our components are compiled sharing runtime and db context with the main app and each other. Consider that then exe can call wdk A and wdk B but A could call B and so on. In this way the connection opened by the exe is the same "passed" to all the wdk.

In this environment, when you update a wdk without changing in / out parameters you do not need to recompile all the other projects, just change and update the wdk on the customer's environment. If you change the analysis removing or changing fields you need to recompile all the sub-projects using the modified datafile. If you add a field you need to recompile only the wdk that will use the new field.

It's not like, but almost like, an exe calling DLLs.
In order to achive and keep this the project model has to be very strict. If you forget or break the rules the runtime will crash..
It's up to you choosing if this is right for you or not. I prefer this rather than mantaining all application modules inside a huge monolithic project.
Posté le 27 juin 2017 - 01:07
Stefano, thank you for your reply. This is much closer to what I was thinking of. Is this using internal components? Or external components?
Posté le 27 juin 2017 - 09:52
External. Internal will bring you to monolith project
Posté le 27 juin 2017 - 13:44
And of course, each time you change the analysis, you need to recompile ALL the components using that part of the analysis (if you can keep track correctly) or ALL the components if you can't, each independently (as each component requires a separate compilation process), and install them all at the correct customer....

Much more complex and prone to mistakes in my opinion than one project with options.

And also, don't forget that if you decide to change an int on 2 to an int on 4 (by example) in your analysis, then you will need to find all the components using that item to eventually change things everywhere, without forgetting any...

This is the process we were going through in the old days, when there were limits on the size of program we could build, and it's really painful.

Best regards
Posté le 27 juin 2017 - 13:51
Hi Art,

I can see your point, and when coming to Windev I also spent a lot of time trying to figure out how to divide the totally integrated system into small, separate and independent subsystems. And that is what I also did with my previous development tool, Clarion CPD, but only because I had to, due to the memory limitations of the DOS operation system; I had a menu program which called some 20 separate subsystems. They all had their own data dictionaries, and whenever a file description was changed, it had to be changed in practically all the subsystems too. Big fuzz.

So I am glad I now have the freedom of the current Windows environments with practically no limitations to the size of the app, and for the reasons related to the integration and analysis, already explained above also by Fabrice, I have kept my system in "one piece".

If you can charge for updates, then there should not be any unsurmountable problems with the delivery of them.

Still, subsystems would be nice, and as I am no programming guru, I would also like to learn in more detail how to divide a totally integrated Windev ERP system into small, separate and independent subsystems.

Best regards
Ola
Posté le 27 juin 2017 - 16:58
Hi Fabrice, Peter, Ola

This is the response from the original replies from Fabrice and Peter about depending on the Analysis.

"So, except for the rare cases where no file is shared between subsystem, I strongly disagree with your microservice approach."
"I'm with Fabrice on this one: Near to impossible with WX if you're depending on a WX analysis."

Like Ola I would like to tame an overly expansive monolithic program and all it's issues.

What I envision is a DATA component/module/exe that would be the ONLY access to the data. Essentially wrapper around the database. Within this wrapper are the functions needed to return data to other calling modules.

For instance, the Customer DB has CustID, FName, LName, PhoneNumber. The functions would be Cust.GetFName, Cust.GetLName, Cust.GetPhoneNumber. They would be called from other modules as res = DATA.Cust.GetFName(CustID), res = DATA.Cust.GetLName(CustID) and res = DATA.Cust.GetPhoneNumber(CustID). The return data coming as JSON, XML or other type of data transport (I have a preference for the system that Fabrice has in his WxReplication program).

If later down the road a module needs to have a middle initial and get the entire name, a Customer.MI can be added to the DB, Cust.GetMI and Cust.GetFullName added as functions, and modules could call res = DATA.Cust.GetMI and res = DATA.Cust.GetFullName to grab the required data from the DB.

In my opinion this is not new stuff, it's simple OO programming. Everything is an object with specific entry/exit points.

To me this would make programming of the non-DATA 'standalone' applications data agnostic. No reliance on the Analysis, you just need a list of the functions from the DATA module that access the data.

None of the non-DATA modules would have any idea what the DB looks like, they would only interface with data through the DATA module. No need to recompile any of the non-data modules when a DB is modified. Of course once a DATA functions are written and used in non-DATA modules the interface must never change. Added to yes, but never changed. Requires a little thought ahead of time but not onerous given the goal of splitting up a gigantic monolithic program.

These non-DATA modules could be swapped out at any time. Disc for Drum, Fuel Injection for Carburetor, no worries about the Analysis at all, no recompiles to include the Analysis.

If the issues is that the non-DATA modules would now be harder to program because, lacking the Analysis, they would not know what the data is (filename.fieldname), thereby making auto-completion of fields impossible. I have a piece of code that I am working on (shamelessly purloined from Peter and modified) that loops through a WDD and creates STRUCTURES based on the Analysis using FileList = HListFile(hLstDetail + hLstAll). (you could probably just as easily create CLASSES - think of this like the Create UML functionality when modifying the Analysis {hope I have that right...}).

This code would be put in ProgramInit, the structures would be created on program startup, put in a .WL that is included during INIT. I would suspect that we could use these structures while programming and it would auto-complete when programming file.fieldnames in code. The structures would also act as the repository for local data fields to put data in that is returned from the DATA module (using the structure as the Record and array of structures for a browse). It's not complete yet, but I don't see why it would not work.

I am not looking for easy or to drink the 10 times faster Kool-aid or "your trying to defeat the reason Wx is so damn great, integration with the Analysis", I just need a manageable way to divide and conqueror the monolith and this is my best idea so far, as imperfect as it is. There must be something I am missing that would totally negate this reasoning but for now it escapes me. I appreciate the insight of our gurus as I am far from being one. Back to y'all for some constructive commentary.
Posté le 27 juin 2017 - 19:05
Art
I can see what you are aiming for and your idea does have some merit but I cannot help thinking you have 'solved' in your reply what would be a relatively straight forward change.

I an assuming here that the Cust.GetMI and Cust.GetFullName methods would return a formatted string to be used/display in a single control so of course this can be done in the DATA class and would be available without the need for changes in any of the components.

But what happens if only a couple of customers requires this middle initial and the others hate it.
Posté le 27 juin 2017 - 20:08
Hi Art,

the approach you are describing is exactly what I was decribing in my first message:

>>...do the data exchange via a webservice that is analysis neutral. So that means a LOT more work...

Now, with that approach, it is POSSIBLE, but much more complex to manage. In that case, you need to update each module accessing a file when that file is changed (without missing any of them) and you will have no automatic system to do it...

If you think that is simpler than letting windev do the work for you, go ahead...

I still think it's a bad good idea :-)
Posté le 27 juin 2017 - 21:40
Well, what Art is describing is provided in large parts by any orm like the .Net Entity Framework combined with its MVC or Hybernate in the java world...

Cheers,

P.
Posté le 27 juin 2017 - 21:54
Derek,

The model I am using is a "three tier model", adapted for a single machine. In "real life" three tier is a local UI (browser usually), usually remote business logic and data from a web service. You are probably familiar with the concept so what I type here will probably be a review, just with my local flavor.

"Separation of Concerns" is another buzz word I like.

On the local machine there is a common DB, surrounded by the DATA module. This is the first tier. This tier has no knowledge of the other two tiers, does not do logic or UI, it's only job is to serve up data. This DB tier is common to all clients.

The second tier is the business logic layer. There are dozens of applications/components for various functionalities in the system (Shipping/Receiving, POS, CRM, Store Config, Reports, etc). This is all code, and the code has no concept of a UI and only gets it data from the DATA module.

We perform constant code improvements as bugs are found or code is factored for efficiency. Each component has no concept of a Wx Analysis as the Analysis is fully encapsulated in the Data Tier. This code is common to all clients, every client has it whether they use it or not, and this code is what will be updated when we want to improve the code from Drum to Disc brakes.

Because it has no concept of the Analysis each app/component is independent and can be replaced at any time. Think short and constant release cycles. Also think that the components will be serialized and conditionally called by a configuration file... that way if the newest release has bugs with a simple config file change we call call the old version.

The common core (business and data layers) are always up to date with the latest bug fixes and improvements, which is critical with PCI DSS and other regulatory concerns here in the US (like OS patches that the client has X amount of days to apply or be in non-compliance).

At any one time there is only one version of the common core. We don't program one version, sell it to the client, then come back later and give them the hard sell to upgrade to the next version. We don't need that headache. (we work on a subscription, you pay x per month and always have the latest and greatest code).

Then there is the UI, the third tier. The only customization we ever do for a client is in the third tier, the UI. This tier is an exe with their name/version/sub version on it (APAL22cUI, for Advance Auto's program, Wx version 22, "c" being the third minor release)(one of our common core modules would be named WR22cCRM, for Wind River, Wx version 22, third minor bug/feature release, CRM module) (you can see how a config file can be used to call WR22bCRM if issues are found in WR22cCRM...).

Each UI is 'unique' and 'custom developed' for each client (ok, we have reusable building blocks of 'common' functionality, but don't tell our clients... they like to think they are special). This is because the UI for the store/website needs to be unique. The client will be replacing another system that has a specific layout and work flow, and we want to mimic as much as possible that layout and work flow to reduce the need for retraining potentially thou$and$ of ca$hier$ and $tore per$onnel (if you $ee the drift there...). One client says the UI has to be simple and intuitive to use, as the folk he hires usually only work long enough to get enough cash for their next fix (such is the state of our inner cities... sigh).

The UI can be branded to the client (logo, colors). We will have a 'stock' Wind River UI for those that do not want or need to mimic their current UI. The UI calls what it needs from the Business Layer. The UI has no knowledge of the DB/Data tier at all and cannot access it.

The corporate side will be less customizable simply because our program needs certain information that is not optional. And they usually pay corporate folks a bit more money cuz they are supposed to be smarter, and there are less of them to train on a new system.

What if they need functionality that we do not currently offer? If the proposed functionality would be of benefit to the majority of our clients, we program it and the DB modifications, move both the DB mods and business logic functions out to all clients in the next minor bug/feature version release.

If the mods are pretty much specific to the client we (for a fee of course) program a DB (hDescribeFile and such) and business logic custom to them, and put them in the customized UI tier. Nothing gets copied to other clients.

So to answer your question... "But what happens if only a couple of customers requires this middle initial and the others hate it." The client has the option of using any, all or none of the functionality contained in the common core. It is like a buffet, chose what you want/need. We will tell all clients ahead of time during development that we will now have the ability to GetFullName and GetMI, if they want to use that functionality then contract with us to modify their UI to call and display it, and if they don't like it, that's ok too.
Posté le 27 juin 2017 - 23:06
Peter,

"Well, what Art is describing is provided in large parts by any orm like the .Net Entity Framework combined with its MVC or Hybernate in the java world... "

Precisely. We just have to work at it 10x slower... :xcool:
Posté le 27 juin 2017 - 23:45
Fabrice,

"the approach you are describing is exactly what I was decribing in my first message:
>>...do the data exchange via a webservice that is analysis neutral. So that means a LOT more work..."

Yes, I saw that and had considered webservices in the past, but webservices require the use of the WAS. I was hoping to dodge that particular bullet. Hence my original question.

"Now, with that approach, it is POSSIBLE, but much more complex to manage. In that case, you need to update each module accessing a file when that file is changed (without missing any of them) and you will have no automatic system to do it..."

Actually none of the modules are accessing a "file". Each module calls a Data.FunctionCall to get/put data. You would only need to update a module that will use the new Data.FunctionCall, not the dozens of other modules that get data from that file.

And since you are going to be in the module anyways that you are making changes to functionality (to get that Cust.GetFullName) we already know we are going to recompile... but not for the purpose of syncing it to the Analysis. Do the fix, recompile and you are done (even if you haven't made any changes to the Analysis yet).

And if it was just a bug fix to the module you don't even have to mess with the Analysis/Data module.

"If you think that is simpler than letting windev do the work for you, go ahead..."

Oh, believe me, it won't be simpler, like you said it is actually more complex and more work. But...

To do it the microservices way may be a royal pain in the neck at development time, having to keep track of what, where and when to change. And it certainly would be easier to let WD do that hard work.

But... if you do it the easy way and build that monolithic program, the maintenance is going to be a royal pain in the ass when you have to roll out that gigantic monolithic program to 5,000 stores just to fix a bug that could have been fixed with a small .wdk/wdl. How many people want to roll out their entire program every two weeks just for some new functionality or a bug fix? That "short release schedule" will start to slip.

With either approach it will involve pain. It depends where you want your pain and how often you want it. :spos:
Posté le 28 juin 2017 - 12:53
Hello Art

if your only problem is deployment, then consider the fact that there is NOTHING to do for deploying a new version, except publish it ONCE on your server (or on a google drive)

With an automatic update system that does NOT need admin rights to work (via the wdl system already described on this forum), you do NOT care about deployment any more...

So building all that for a false problem? I think not!

Best regards
Posté le 28 juin 2017 - 15:27
Hi Art,

I must confess I can't understand all your reasoning, but then, on the other hand, I do not have 5000 shops as customers...

A customer specific feature can be "hidden" in a "monolithic" system; you give its "secret code" only to those who have paid for it.

I have experienced no problems compiling a "monolithic" update package and putting it up on an internet server, automatically. It's rather a question of clicking a button, not of how many users there are.

If there is a need to put up an update every two weeks, then perhaps the system could have been better planned in the beginning.

I can see that programming your scenario would be "a royal pain in the neck at development time", but I suspect it would also be a continuous pain in the sitting muscles during later maintenance, especially for anybody else other than the original developer himself.

IMHO programming should be fun, not painfull, so I think I'll stick to my monolithic "KISS" approach, complete with old-fashioned procedural programming:).

Best regards
Ola
Posté le 28 juin 2017 - 15:53
"If there is a need to put up an update every two weeks, then perhaps the system could have been better planned in the beginning."

Considering Microsoft patches its systems on a weekly basis, and flash every two weeks, you may have a point... ;)
Posté le 28 juin 2017 - 23:28
Hi Art,

Maybe you can take a look at the patching sytem of Windev, you can release a patch that is only the differences between the latest full build and the changes made. It can contain a report, windows, code,...

Get some more info in the help:
http://doc.windev.com/en-US/…

That may solve your update problem ?

Danny
Posté le 29 juin 2017 - 09:39
Hi Danny,

It is... But not when going OO (typically when doing multi tier) where it is useless...
Quote
Help Page

Note: Patch containing classes: If your patch contains a class, the declaration code of the class must not be modified in relation to the version used with the executable. The declaration code must be identical: same members, same structures, same declaration order. Otherwise, the executable must be re-created

Cheers,

Peter Holemans
Posté le 29 juin 2017 - 11:44
Re: Patches

It also seems that if using patches, there is the added task of managing (removing) the obsolete patches from the exe directory. I understand that this can be configured to be handled automatically when using WDInst, but if using some other installation software (eg. InnoSetup), this management must be manually coded.

Maybe somebody can shed more light on this?

Best regards
Ola
Posté le 29 juin 2017 - 13:59
Hello,

I has replied to this post in PCSoft's forum but I don't see my replies there in this forums so please check these links:

https://forum.pcsoft.fr/en-US/pcsoft.us.windev/61449-microservices-61452/read.awp

https://forum.pcsoft.fr/en-US/pcsoft.us.windev/61449-microservices-61459/read.awp

Regards,
Posté le 29 juin 2017 - 23:57
Fabrice, Danny, Ola,

"if your only problem is deployment" and "That may solve your update problem ?"

If the only problem I had with a large monolithic program was an install, I'd be happy.
But apparently my first response to Ola was To Long;Didn't Read (TL;DR) for some people, so I will repost the relevant part here (emphasis added... from direct experience with clients):

"Also, in my opinion, there are inherent drawbacks to using monolithic architecture... it quickly leads to monolithic hell for any large and/or complex application that needs to be modified or improved. I've already had to deal with a large monolithic program called WinDSS (a point of sale program still used here in the US). As retailers have found out WinDSS needed to be modified, sometimes heavily, to fit their business needs.

Adding any new functionality took an inordinate amount of time and was replete with unanticipated bugs and side effects. Cross application changes were a nightmare, and supposedly simple changes affected other parts of the program in ways unanticipated and frequently undetected until it hit the stores.

The complexity buries you quickly once the application grows from it's simple roots to the enterprise level. Integration of applications are difficult except through the database, which I think was Fabrices point.

Over time the code became messier and messier as more maintainers worked with the same code, putting patch over patch until entire sections need to be refactored or completely rewritten (been there, done that).

I have always said WinDSS was like a balloon... push on one area and it bulges somewhere else. But as a WinDSS developer I can't complain to much. WinDSS had made me millions at the expense of multiple retailers."

Been there, done that, didn't even get a tee-shirt. Lessons learned, trying to avoid a repeat by better planning (Ola's point).

EDIT: For Ola, my last two clients I worked for maintaining WinDSS, Cabelas (when I left in 2012) had 45 stores with an average of 30 registers each, and Advance Auto had over 5,000 stores with an average of 4 to 5 registers each.
Posté le 30 juin 2017 - 18:13
Hi Art,

"As retailers have found out WinDSS needed to be modified, sometimes heavily, to fit their business needs."

So some of your clients clearly made a wrong choice when purchasing their software, either because they could not originally define what they needed, or your salesman was only interested in getting the order.

And then you had to try and a make a Mack truck out of an MX-5:)

I have turned away some really juicy customers, because I knew my app would not serve them well, and I would get unnecessary trouble trying to keep them satisfied.

"Integration of applications are difficult except through the database" is also my point, and it is easiest to achieve in a "monolithic" app. Without a common database any integration may easily become a mess kept together with wire and chewing gum.

Best regards
Ola
Posté le 01 juillet 2017 - 19:31
Hi Ola,

"So some of your clients clearly made a wrong choice when purchasing their software, either because they could not originally define what they needed, or your salesman was only interested in getting the order."

With a few more insights like this, we will grant you an honorary American Capitalist certificate and welcome you to the club!

"And then you had to try and a make a Mack truck out of an MX-5"

My comparison to coworkers is we tried to put a Mack dump truck on top of a Toyota Tacoma frame.

"I have turned away some really juicy customers, because I knew my app would not serve them well"

HERESY! Your honorary American Capitalist certificate has been revoked!
Posté le 02 juillet 2017 - 20:45
:xcool: