feat: 🔥 First punt at priority option

[theryansmee]

Allow user to set length priority to length or unique (POC)

✅ Closes: #220

PR Checklist

Please check if your PR fulfills the following requirements:

• The commit message follows our guidelines: https://github.com/ngneat/falso/blob/master/CONTRIBUTING.md#commit
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

[theryansmee]

@NetanelBasal @shaharkazaz - This is a super sloppy implementation that I threw together at lunch today, what are your thoughts?

[NetanelBasal]

Thanks @theryansmee, @shaharkazaz will go over it asap.

[shaharkazaz]

🚀

[theryansmee]

🚀

#223 (comment):
Yeah you are right. I'll have a think about how we can compare complex objects. With most of our entities be happy to consider it uniques as long as the id is unique.. but i'll have to look through our functions to see what other challenges we face.

I've fixed the other bits

[theryansmee]

@shaharkazaz @NetanelBasal I've still got to write tests and clean up a few bits, but I think this is mostly there now. What are your thoughts? 😊

[shaharkazaz]

@theryansmee I'll re-review it again today 👍

[theryansmee]

@shaharkazaz I've ended up creating objectIsUnique which is then used by the other checker functions around the library. Originally, I was hoping to just have a single function that got passed around by I ended up in type hell.

That said, I think this is actually quite an elegant solution.

PS
I think once we are happy with this MR, we should do some follow up work on splitting core.ts into multiple files and ensuring that it has a decent test coverage.

PPS
I'm not sure why the build it failing. Looks like we have an issue with packages somewhere down the line.

[theryansmee]

🙃 Just seen theres been commits to core.ts so i'm going to have to look at what they are and get them integrated into my work

[NetanelBasal]

Cool, thanks!

[theryansmee]

I will be fixing the conflicts and merging this later this week :)

[theryansmee]

@NetanelBasal - I've merged in the latest changes but now the build is failing on the setup-node@v2 step with getCacheEntry failed: Cache service responded with 503.

Any thoughts on what might be happening?

[NetanelBasal]

Nopehttps://stackoverflow.com/questions/72598852/getcacheentry-failed-cache-service-responded-with-503

[theryansmee]

@NetanelBasal @shaharkazaz - Hi guys, sorry for the radio silence from me (Life got a little hectic). I have fixed the merge conflicts and added a few essential tests to prove priority logic works as expected with data from arrays and from functions.

I think it's ready to be reviewed again 😊

[shaharkazaz]

@theryansmee Hi Ryan always a pleasure to hear from you 😁
I'll re-review it today-tomorrow.

[theryansmee]

@theryansmee Hi Ryan always a pleasure to hear from you 😁 I'll re-review it today-tomorrow.

I'm back for good this time! haha Cheers mate 😊

[shaharkazaz]

This is the first cycle, I'll go in-depth over the core files, these are the first things that I saw

[shaharkazaz]

In the case of an object, you always pass comparisonKeys right?
If so, let's make this type stricter, and the values should also be a sub-set of T's keys.

[theryansmee]

If i'm honest I was having a little trouble with the types for this. I had a lot of issues when I tried to make this not optional. I have changed the comparisonKeys type from string[] to (keyof T)[] though

[shaharkazaz]

Why any and not a generic?

[theryansmee]

Yeah I wasn't a fan of this either. On further inspection i've managed to change it to objectIsUnique<T extends Record<string, any>>. I think this should be ok for now

[shaharkazaz]

Same

[shaharkazaz]

Same

[shaharkazaz]

I agree with Netanel's previous comment about the naming.
Maybe something like:
it(Should pass when object ids are unique) instead of the describe. This goes to all the specs I've seen in the recent files.
@NetanelBasal thoughts?

[NetanelBasal]

Agree

[theryansmee]

This is the first cycle, I'll go in-depth over the core files, these are the first things that I saw

Cheers mate, i'll have a look through and get these sorted tomorrow 👍

[theryansmee]

@shaharkazaz - Hello mate, I have made all of the requested changes and left a few comments 😊

[shaharkazaz]

You are handling the length options both here and in the fake function, please choose one place

[shaharkazaz]

I think it makes sense to pass the attempts as the function's argument.

[theryansmee]

@shaharkazaz - How do you mean? To allow the user to decide the total number of attempts?

[shaharkazaz]

You are passing 0 to that factory function, I think it makes more sense to pass the attempts, if the factory is using the index passed than you will increase the chances of getting the same result.
@NetanelBasal WDYT?

[NetanelBasal]

Agree

[shaharkazaz]

same comment as above

[shaharkazaz]

There is a hidden assumption here that the array you are given holds unique values.
Please change the code to support non-unique arrays.

[theryansmee]

@shaharkazaz - You are correct. How did I not spot that :/

I have reworked it to shuffle the data array and then pick the first n items from it. A positive of doing it this way is that the user will receive an array with a length as close as possible to their desired length. The issue is that this will be more expensive for large data sets.

export function fakeFromArray<T, Options extends FakeOptions>(
  data: T[],
  options?: Options
): T | T[] {
  if (options?.length === 0) {
    return [];
  }

  if (!options?.length) {
    return randElement(data);
  }

  const priority = options?.priority ?? 'length';

  if (priority === 'length') {
    return Array.from({ length: options.length }, () => randElement(data));
  }

  let clonedData: T[] = JSON.parse(JSON.stringify(data));
  // Randomise order of data
  clonedData = shuffleArray<T>(clonedData);

  if (options.length >= clonedData.length) {
    return clonedData;
  }

  return clonedData.slice(0, length);
}

function shuffleArray<T>(array: T[]): T[] {
  for (let i = array.length - 1; i > 0; i--) {
    const randomIndex = Math.floor(random() * (i + 1));

    [
      array[i],
      array[randomIndex],
    ] = [
      array[randomIndex],
      array[i],
    ];
  }

  return array;
}

What are your thoughts?

[shaharkazaz]

You have a lot of shared code here, I think I would normalize the array data source to also work on a function with a random element and just use factory-based fake for both.

[theryansmee]

I'm happy to do this I just felt like this whole fake logic was getting quite complex so keeping them separate made it easier to read for me. But i'm happy either way.

I think in hindsight, if we were starting over i'd suggest that instead of option.length property we have singular and plural function for each. E.G randId that returns a string & randIds that uses randId to generate & return a string[]. But it might be a bit late for that ;) haha

[shaharkazaz]

Personally I prefer the length property, adding a plural will double the API for not a good enough reason from my point of view

[theryansmee]

Fair play. It works pretty well for our users, it just adds a lot of headaches for deving haha. All good though

[shaharkazaz]

This is repreated several times, maybe create a factory function that creates the chacker

function uniqueByKeys(keys){
  return (a, b) => objectIsUnique(keys)
}

[theryansmee]

@shaharkazaz - Sorry I don't know how I missed this. Do you mean something like:

export function randCreditCard<Options extends CreditCardOptions = never>(
  options?: Options
) {
  const factory: () => CreditCard = () => {
    const fullName =
      options?.fullName ??
      `${randPersonTitle()} ${randFullName({ withAccents: false })}`;
    const brand = options?.brand ?? (randCreditCardBrand() as Brand);

    const dateOptions: Intl.DateTimeFormatOptions = {
      month: 'numeric',
      year: '2-digit',
    };
    const validFrom = randPastDate({ years: 1 }).toLocaleDateString(
      'en-GB',
      dateOptions
    );
    const untilEnd = randFutureDate({ years: 2 }).toLocaleDateString(
      'en-GB',
      dateOptions
    );

    return {
      fullName,
      brand,
      validFrom,
      untilEnd,
      ccv: randCreditCardCVV(),
      number: randCreditCardNumber({ brand }),
      account: randAccount(),
      type: rand(['Credit', 'Debit']),
    };
  };

  return fake(
    factory,
    options,
    {
      // Moved logic into here rather than wrapping objectIsUnique call in new function
      uniqueComparer: (card, cards) => objectIsUnique(card, cards, ['number']) 
    }
  );
}

[shaharkazaz]

Isn't this checking the opposite? Don't you need all jsons to be different?
Looking at this as a reference:

[shaharkazaz]

Same comment as the json unique

[theryansmee]

@shaharkazaz - I have made the requested changes for all the comments that I understood and left you queries everywhere else :) I hope these all make sense

[theryansmee]

Ok, I think all of the requested changes have been made with just a few outstanding queries left to be answered

• For @shaharkazaz: feat: 🔥 First punt at priority option #223 (comment)
• For @shaharkazaz: feat: 🔥 First punt at priority option #223 (comment)
• For @NetanelBasal: feat: 🔥 First punt at priority option #223 (comment)

[shaharkazaz]

@theryansmee I'm actually on vacation in Thailand 😀
I'll get back to it as soon as I return!
If @NetanelBasal is free he might pick this up