Extensible analytics

Access to execute Python code for taking the inputs (variables) and performing data intensive calculations will be very helpful. Currently, I use PI Web API and use Task Scheduler to fetch data from PI AF to perform more advanced calculations. If we can call a custom Python function, that will be a good start. Pandas library for instance can achieve a lot with AF attributes as inputs.

Would you please expand on this idea little bit? Do you mean something like custom/predefined Expression functions, or rather full programmatic access?

This overlaps with the advanced analytics/extensible analytics. However, I see a slight difference. What I have in mind is that we would be able to create a function similar to the pre-built functions that can be used in Analytics. I would like to see this in two forms: 1) take a set of AF Analytics and expose this as a function. Basically you would build an Analytics template in PSE that is then exposed as a function in the Element Analysis tab. 2) build a function using your favourite programming tool (C#, F#, ....) which is then available in AF Analytics like a normal function.

Because the ACE wizard isn't working anymore in VS2015, it would be good to get away from ACE, and make use of the AF Analysis scheduler. https://pisquare.osisoft.com/message/94038-re-ace-wizard-and-visual-studio-2015?et=watches.email.thread#comment-94038

This request could go a long way, so maybe this option solves the request in a more broad sense: https://feedback.osisoft.com/forums/555148-pi-server/suggestions/17219900-extensible-analytics

I'm a bug fan of AF, yet every decent project i needed to create my own external calculation engines for more complex business logic. I've used SQLserver, OLEBDEnt and linked AF tables, or custom C# engines. In all cases i would at least get rid of all the technical plumbing of management, scheduling, security, etc., just like the ACE scheduler was. So mechanisms to trigger custom logic, supply the data and provide access, and send back the data to AF. If needed, i can supply specific examples i've built for customers.

i think this request is more specific on the external calculation: https://feedback.osisoft.com/forums/555148-pi-server/suggestions/17219900-extensible-analytics

Let me add a current example where we use a custom C# calculation engine for. For each batch, and during the batch at every change of the batch phase, we need to calculate the operating window for a number of key process parameters. We trigger the engine based on signups to BatchID and PhaseID. On every change, we lookup some process data, and some AF table, calculate our operating window limits (data series using future data) and write those back to PI Points.

One main concern here is that if we have periodic calculation occurring every minute, then as of now with around 5k AF analyses, some calculations are being calculated with delay. During this duration of delay (generally 5-7 seconds) value will come as calc failed. So we need to take in concern that also.

I liken this to the custom function builds in Excel. Simple code defines the inputs and calculations. The function appears in the library and can be used in analytics, i.e. customfunction('attribute1','attribute2','time1','time2') This would be supplemental to using templates, allowing more flexibility as to where / when the function could be used.

A quick solution could be, providing a Analytics Function to trigger an external script from a script folder. By default when the script is triggered, it should provide the Asset Context & Time Context to the script, Also it should have additional placeholder to pass custom information to the script. Then Write you script in Powershell, Python, R; use AFSDK, WebAPI, SQL DA, as per one's comfortably.

I've worked with customers who have expressed the same desire. They want to do some basic pre-calculations on many of the incoming data streams across the AF database, e.g., quality checks like badval or range limits and/or some basic pre-processing like moving average with a time window defined in an attribute. It can get tedious having to copy paste all of these lines of expressions from one template to another for many attributes. It would be easier to build a predefined function of their own where they just need to add the attribute name and few parameters like time range. There have been requests for both user defined functions that only use a set of the included Analysis functions (this might be a good first step) and custom functions written in some programmatic language similar what was done in ACE.

Customer would like to have the following possibility: During the installation of a new Analyses service kit pointing to a PI AF server where an analysis service is already configured, he would like to prevent the current PI analyses service from stopping during the installation so that no real-time data calculation is lost. He suggested this because he would like to keep his current analysis service running without forcing it to stop despite this new installation.

I appreaciate the step back! While Matlab provides scaleability performance and capabilities on the high-end of the spectrum, there is still a demand on the lower end. I do understand the complexities and pitfalls involved (remember Custom DR's), but there should / could be a midway, just like PI Vision seems to strike a midway with it's extensibility. E.g. leaving the data delivery to the custom function to the analysis service, but having the capability to use e.g. C# to implement custom functions within that limited frame.After all we don't want to go back to the days where we open, render and parse Excel files in an ACE module...

Can you provide a bit more details? When you say "...Advanced Calculation feature/product ...", I'm not fully understanding what you're describing. Can you provide an example use case?

Now that Matlab is off the table, any new directions for extensibility of Analytics?

Can this suggestion be re-opened? Now that the MATLAB Production Server is being pulled back.

We would like to be able to add custom analytics that can be exposed in the same way the Matlab functions were, effectively passing input data to an external calculation engine and then return results to AF Analytics. This would allow us to migrate many legacy ACE calculations to AF Analytics for a number of clients, where we are currently performing complex calculations that are effectively contextual to a specific AF Element. In quite a few of these cases, we have used ACE as a scheduling container, but the code is typically AF SDK in order to interact with the asset hierarchy.

Hi,   since the integration solution is still up and explained on the MATLAB web site (!), it would be interesting to learn more about why the integration was withdrawn from PI AF 2018 SP2.   We were only a couple of weeks away from buying the MATLAB Production Server...   Regards, Tomas

In response to Tim Carmichael, "Since I haven't had an opportunity to us..." Hi Tim Carmichael   Data References always work on-demand, but an Analysis has the benefit of persisting the output to PI point for history and faster retrieval.  The issue is there are some things that can't be done in Asset Analytics since it lacks a true looping mechanism or conditional branching.  A data reference may access a 3rd party library but configured analytics cannot.   The reason for the proposed idea is to have a standard hooks from Analytics to a calculation service of the customer's choosing (not just MATLAB).  In this use case, Analytics is regulated to be the scheduling service but it shells the actual calculation to an external service.