[feat] Add benchmark #1977

Activity

1. loynoir commented on Apr 5, 2023

   

   loynoir

   on Apr 5, 2023

   Last edited by loynoir

   Author

   More actions

   Done.

   https://github.com/loynoir/git-bench/blob/main/bench.txt
2. loynoir

   closed this as completedon Apr 5, 2023
3. axkibe commented on Apr 7, 2023

   

   axkibe

   on Apr 7, 2023

   More actions

   Any idea why simple-git is so much faster than execa than execFile? Effectively it should be about the same, albeit of course spawning a process will always be slower than a native library. Also isomorphic.. I guess the first time V8 JIT compiler will take a lot of time, but it's still consistently slower over many calls?

   Also anyone ever tried to compile canonical git using emscripten to webassembly? I wonder how that one compares in benchmarks.
4. loynoir commented on Apr 8, 2023

   

   loynoir

   on Apr 8, 2023

   Last edited by loynoir

   Author

   More actions

   @axkibe

   You remind me to test whether /tmp is tmpfs, and found devcontainer /tmp is not tmpfs.

   Now I

   • Add code to ensure BENCH_REPO is under /tmp and /tmp is tmpfs
   • Change BENCH_COUNT from 100 to 10000
   • Unfortunately, after devcontainer rebuild and system update, isomorphic-git encounter bug 1163, and remove from bench

   https://github.com/loynoir/git-bench/blob/733d6b7c7b3bae9f5655396b023e2889b302a4ae/bench.txt#L1-L8

   Seems simpleGit is still a bit faster.

   Have not look into simpleGit code, I guess raw git command parse config every time, and simpleGit somehow cache config.

   Just a reminder, if you want to use simpleGit, simpleGit_show() return string if I remember correctly, the other XXX_show() return Buffer.
5. sosuisen commented on Apr 8, 2023

   

   sosuisen

   on Apr 8, 2023

   More actions

   @axkibe

   Also anyone ever tried to compile canonical git using emscripten to webassembly? I wonder how that one compares in benchmarks.

   I have found a wasm version of libgit2, though not the canonical git.
   I have not yet examined it closely, but I feel it is promising.

   https://github.com/petersalomonsen/wasm-git
6. axkibe commented on Apr 8, 2023

   

   axkibe

   on Apr 8, 2023

   More actions

   Out of some reason, isogit is not looking for the commit object in the pack file, this "fixes" it as workaround in berry:

   mv .git/objects/pack/pack-5594355c4b021332ab59e14ea79d5c46bf10ec68.pack .
   git unpack-objects < pack-5594355c4b021332ab59e14ea79d5c46bf10ec68.pack

   of course not a really fix, but it can run the test:

   ┌─────────┬──────────────────────┬────────────────────┬────────────────────┐
   │ (index) │      Task Name       │ Average Time (ps)  │   Variance (ps)    │
   ├─────────┼──────────────────────┼────────────────────┼────────────────────┤
   │    0    │    'nodegit_show'    │ 14269.009348071486 │ 1375.234011865769  │
   │    1    │ 'isomorphicGit_show' │ 48132.35018748002  │ 20944.06588460185  │
   │    2    │   'simpleGit_show'   │ 33357.35877334451  │ 5235.988108480814  │
   │    3    │   'execFile_show'    │ 32022.69114060595  │ 15853.465081924764 │
   │    4    │     'execa_show'     │ 30753.440147191515 │ 2675.103913497697  │
   └─────────┴──────────────────────┴────────────────────┴────────────────────┘
   

   But also not reassuring for production use, considering no one there can actually fix it.

   PS: I tried to compile canonical git with emscripten.. it compiles, and kinda works, but also doesnt with lots of tiny and issues due to the file system simulation, but eventually guess it was a stupid idea, because it's forced to use sync io and there is no way around that.