More functions to support operations on arrays

Functions like mapdata would be useful: A series of values at one time is possible. Array operations should be able to write arrays back. Quality systems work with arrays of data.

Here are a few helpful functions with arrays that could add a lot of value:

1. It would be useful to allow a random sampling of values from an Array and use a Boolean switch to allow\not allow repetitions. As done by the "sample" function used in R.

   The reason is that often in manufacturing one needs to take a random set of an array and in the current PI analysis functions available this is not possible. (or at least some compromises must be taken which does not make the sample truly random)

   Here is an example:

   1. Lets say we use the "Recordedvalues" function in PI Analysis to retrieve a set of PI tag values for a given time interval as an Array.

   2. We would like to sample N values from the above Array randomly WITH NO REPETITIONS (the same value cannot be sampled more than once) since our goal is to calculate a few aggregate values based on that random sample size (average, minimum, maximum, etc) and save the outputs into PI Points.

   3. Currently, a few options to do so with the available PI Analysis functions are as follows: We can generate a random number using the Rand() function as an INDEX and then extract the value from the Array using the generated random index - This method though does NOT guarantee that there will be no repetitions. Or we can choose a fixed sample size for all calculations and using it as a divisor for the array length we can calculate a step value and traverse the array to extract values: 1, 1+step, 1+2*step.... etc up to N and push them into variables\PI Points. The last method is of course far from Ideal since the sample size is fixed and as such it cannot capture true random variability in the process and may inadvertently sync with periodic patterns in the process or miss critical process variations that happen to fall between the fixed steps.

2. Additional useful Array functions:

   1. It would be useful to be able to "shuffle" an array randomly (could provide a solution to the above use case as well as other use cases)

   2. It would be great to be able to create an array of values similar to the Sequence function used in the R programming language.

thanks.

There are some more suggestions: More linear and bi-linear interpolation support, polynomial calculation using arrays as coefficients.

Asset Analytics was designed to be a streaming calculation engine, therefore it supports very well calculations that executes on a schedule, in a streaming fashion. These suggestions on linear interpolation and polynomial calculations seem to be more suitable for "adhoc" calculations, ones that run only when needed. Can you provide additional details on what your specific use cases are - i.e. it would help me if you can provide a high level description of what problems you're trying to solve. Thanks.

Need capability to perform dot products , i.e. A1 = [2,3,5], A2 = [4,5,10] then A1*A2 = [8,15,50] Also a Sum of all the members of the array , Sum(A1) = 10

Thanks David.  I like the more generic dot product

Ales Soudek You can already do Total(A1) to sum an array.   For dot product, I have been thinking something more generic could work even better  (not implemented): MapData(A1, $val * A2[$i]) - basically, besides exposing $val, also expose the index $i. Just an idea for now.

I would need a possibility to count distinct values. My Usecase: I tried to implement some sort of Controller Performance Monitoring in AF. Part of this monitoring is the calculation of Output and Setpoint Changes as well as a service factor of the controller. For us it is relevant if plant operators use controllers in manual mode or if it is operating in automatic as it should be. Therefore we want to monitor how often the output was changed when operated in manual and how often the setpoint changed when in automatic mode. As a first shot I created some event-triggered calculation counting the output and setpoint changes dependent on the controller mode. But as we had some performance issues with analysis service running on event-triggered calculations, this is not what I want to rollout for 20.000 controllers. So I tried to do the same, using Arrays of the recorded values of one day and managed to filter the interesting values (FilterData(Setpointvalues,TagVal('Mode',TimeStamp($val))="AUTO"). But now I can't proceed any further as I will receive a dynamic length array with an unknown amount of same or different values ([T1:V1,T2:V1,T3:V1,T4:V2] - Timestamp and Value). There is no way for me to find out how many different (distinct) values there are.

In response to Stephen Kwan, "I think there are simpler way to accompl..." That is what I tried but array.length is only correct if there are only distinct values in the array. But if I got the recorded values of a day, chances are high to have the same value with different timestamps. For example due to compmax of 8 hours. If I only want to know how many changes in value have been that day, array.length will result in 3 (12 am, 8am, 4pm) although there have been 0.

In response to Jürgen Schmidt, "I would need a possibility to count dist..." I think there are simpler way to accomplish what you need. Variable1: FilterData(Setpointvalues, $val = "AUTO") Variable2: ArrayLength(Variable1)   The first row sets your array with values that are "AUTO".  The second row gives you a count.  The ArrayLength function would handle the different array lengths.

Ok, so you really need to count only if there are changes. Do you wish to do this counting in an adhoc basis (reporting or basically on demand query) or in a continuous basis?

In response to Stephen Kwan, "Ok, so you really need to count only if ..." Our Controller Monitoring by now is based on a daily report (created once a day at midnight) in a third party application. We are thinking about changing this to a reporting of the last 24h either when a user requests data or calculated every hour (if possible) but would be happy already, if we could manage to implement the one time per day report in AF.  So we need to create a retrospective counting either for timerange begin of day(yesterday) to begin of day(today) or now to now-1d.  

In response to Jürgen Schmidt, "Our Controller Monitoring by now is base..." Is this a digital tag?  If it is not a digital tag, is it configured with "Step" = 1?

In response to Stephen Kwan, "Is this a digital tag?  If it is not a d..." In our case right now, it is about the setpoint of a controller. So no digital tag but a tag with Step=1.  However we are also thinking about creating an analysis how often the direction of the controler output changed - here it is not about the number of changes but about how often did the output of a controler change from closing a valve to opening a valve and vice versa.  A foreach loop to access the data would be nice: ForEach(val[i], Array,Function)

Adding my comment and to this - the "dotproduct" suggestion below is one I'd love to see (array multiplication, or equivalent of Excel SumProduct (which can be implemented as DotProduct followed by a Total). I thought that MapData(Array1,$val * MapData(Array2,val$)) might work, but it can't. The use case for this for me is where we (for example) store related data in two arrays (eg. prodct flow and product yield), and want to determine the overall sum. I also see a significant benefit in being able to implement some correlations - eg. compressibility factor for gas compositions at P&T; coudl even allow a full calculation of meter performance (GHV, Z, SG, Mol Wt etc) against a composition.

In response to Andrew Inwood, "Adding my comment and to this - the "dot..." David Pugal's idea of exposing the index inside the MapData function would allow something like this:   MapData(Array1,$val * Array2[$i])   I'd like to see functions for searching an array and returning the index of the match. I have a use case where I have one array of events, then a second array with the results of a calculation based on the events in the first array and I want to choose one of the event values from the first array to output based on comparing a desired value to the results in the second array.

It looks like another classic PE function doesn't work with arrays either: IsSet() Example: 'ArrayAttribute' - 5 tags, one of them has a value with "Questionable" flag in its snapshot event. FilterData('ArrayAttribute',NOT IsSet($val, "q")) returns 5, even though it should return 4 instead. Also if the questionable event happens to be in the last tag in the array, the following function would fail too: IsSet(LastValue('ArrayAttribute'),"q") --- this one would result in False, but it should be True.

Sorting arrays would be useful as well. This would further allow for calculation of percentiles.

Really Important