@actyx/pond: observeBestMatch shows some odd behavior

Description
I’ve tried to use pond.events().observeBestMatch() to find the event that happened directly before or on a given timestamp. In other words: I’m looking for the event with the highest timestamp that is still smaller or equal a given value. For example I have 10 events whose timestamps have the ascending values 0, 1, 2, 3 and so on… Now I want to know the event that has the highest timestamp but is still smaller than or equal to 5. I expected observeBestMatch to determine the 6th event as best match as it has the timestamp value 5. I used this as my shouldReplace-function:

(nextCandidate, currentBest) => {
              return nextCandidate.meta.timestampMicros >
                currentBest.meta.timestampMicros &&
              nextCandidate.meta.timestampMicros <= 5;
          }

Expected/Actual behavior
I expected the function to determine the event with the timestamp 5 as the best match. It actually
determined the event with the highest timestamp (9) as the best match.

Reproducing the error
I’ve created a repository showing the unexpected behavior with failing tests in Jest:

You just need to run npm install and npm run test.

Interestingly, when printing the events that come in as candidate and cur into the shouldReplace-function, they seem like they should be the other way 'round.

Another Thing
When using observeBestMatch, tests with Jest still report unclosed handles, even though I discarded the pond and closed the subscription. Not sure what this is related to.

Configuration
@actyx/pond": “^3.0.1”

Hi sam.lk!

There is no specific guarantee regarding the order in which candidates are passed to your callback. Hence, your callback must always select the better match, irregardless of what it has seen so far.
(Note that this is intentionally different from Fish’s onEvent which has a guaranteed order. The point about observeBestMatch is to enable a performance gain in cases where you don’t need the Fish’s “time travel” behavior.)

The example you have posted is very sensitive to the order of events:
shouldReplace(5, 9) wrongly results in false (sticking with 9), because the first condition fails.
shouldReplace(9, 5) correctly results in false, sticking with 5.
(Using numbers as shorthand for “event with this timestamp”)

Changing your code like this:

const distance = (e: ActyxEvent<unknown>) => Math.abs(targetTimestampForBestMatch - e.meta.timestampMicros)
// …
const shouldReplaceValue = distance(nextCandidate) < distance(currentBest)

… makes the test pass.

Does that make sense to you? I will enhance the documentation a bit to try and make it clearer that the function must not care about order of events.

When using observeBestMatch, tests with Jest still report unclosed handles, even though I discarded the pond and closed the subscription. Not sure what this is related to.

This is a bug, thanks for spotting it! Will be fixed in the next release.

Best regards,

Benjamin

Thank you Benjamin for the explanation!
Your proposed solution makes the test run fine.

Although it still puzzles me a bit why the current-best changes when shouldReplace returns false.

    Current best: 1, Potential next candidate: 0, shouldReplace returns: false
    Current best: 2, Potential next candidate: 1, shouldReplace returns: false
    Current best: 3, Potential next candidate: 2, shouldReplace returns: false
    Current best: 4, Potential next candidate: 3, shouldReplace returns: false
    Current best: 5, Potential next candidate: 4, shouldReplace returns: false
    Current best: 6, Potential next candidate: 5, shouldReplace returns: true
    Current best: 7, Potential next candidate: 5, shouldReplace returns: true
    Current best: 8, Potential next candidate: 5, shouldReplace returns: true
    Current best: 9, Potential next candidate: 5, shouldReplace returns: true

… but I guess that’s an implementation-specific thing.

Still, your solution would get me the event that’s closest to the given timestamp which can be either before or after the timestamp. But I only need the event that happened immediately before the timestamp. For my use-case I would also need to select which streams I want to include in the best-match-operation, so I guess the function is just not suited for that (since you cannot pass an OffsetMap as query parameter).

Ah, got it now. The argument order is indeed reversed while traversing historic events. It doesn’t make a difference as long as
shouldReplace(a, b) <=> !shouldReplace(b, a)
holds, because the logic is also swapped.
But I agree it’s not good. Will be fixed in next release!

Still, your solution would get me the event that’s closest to the given timestamp which can be either before or after the timestamp. But I only need the event that happened immediately before the timestamp.

In that case, make it:
const shouldReplaceValue = nextCandidate.meta.timestampMicros <= targetTimestampForBestMatch && distance(nextCandidate) < distance(currentBest);

If there is no candidate matching condition <= targetTimestampForBestMatch, you will get a random result. Otherwise you will get the closest <= 5, as desired.

Cheers!

1 Like