| Svelte Hacker News

burntsushi 9 years ago

My guess is that since you ran `rg` first, the file wasn't in memory, and you ended up benchmarking disk IO. (Notice the sys time decrease from your first run to the second run.) Subsequent commands then run faster with the file already in memory.

This is one of many reasons why assembling the benchmarks in my blog post was so difficult. For example, on every command I benchmarked, I ran them 3 times for "warmup" and didn't record any measurements. I then ran them another 10 times in which I recorded them. You can see the raw output here: https://github.com/BurntSushi/ripgrep/blob/master/benchsuite...

In any case, on my underpowered Mac, here are some results on a 1.2 GB file (notice how much the time fluctuates until its fully in cache):

    mac:~ andrew$ ggrep --version
    ggrep (GNU grep) 2.25                                                                                                                                                                                                
    Packaged by Homebrew
    mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null   
    
    real    0m5.447s
    user    0m0.600s
    sys     0m0.350s
    mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m1.247s
    user    0m0.549s
    sys     0m0.264s
    mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m0.803s
    user    0m0.542s
    sys     0m0.259s
    mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m0.805s
    user    0m0.544s
    sys     0m0.260s

And now for rg:

    mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m1.062s
    user    0m0.339s
    sys     0m0.333s
    mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m0.640s
    user    0m0.337s
    sys     0m0.302s
    mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null
    
    real    0m0.637s
    user    0m0.336s
    sys     0m0.300s

Oh! And check this out, on a Mac, not using a memory map for single files is faster. My goodness---memory map performance is all over the place.

    mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl --no-mmap > /dev/null
    
    real    0m0.445s
    user    0m0.170s
    sys     0m0.274s

If I do this on my Linux machine on the same file, I get timings of 0.275s for rg, 0.398s for rg with no memory maps (opposite direction for Mac) and 0.708s for GNU grep (v 2.25).

Benchmarks are fun, eh?

atombender 9 years ago

I ran each test four times and picked the best result — not my first rodeo — but for the first result I picked the wrong time from my output, which obviously didn't make use of the cache. Here's it again, complete result, added --no-mmap:

    # ggrep
    zerogravitas$ for x in {1..4}; do (time ggrep erzg4 k.json >/dev/null); done
    ggrep erzg4 k.json > /dev/null  1.96s user 0.67s system 99% cpu 2.641 total
    ggrep erzg4 k.json > /dev/null  1.95s user 0.68s system 99% cpu 2.660 total
    ggrep erzg4 k.json > /dev/null  2.00s user 0.66s system 99% cpu 2.672 total
    ggrep erzg4 k.json > /dev/null  1.96s user 0.67s system 98% cpu 2.662 total

    # rg
    zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json >/dev/null); done
    rg erzg4 k.json > /dev/null  1.76s user 1.40s system 99% cpu 3.180 total
    rg erzg4 k.json > /dev/null  1.77s user 1.31s system 99% cpu 3.088 total
    rg erzg4 k.json > /dev/null  1.74s user 1.36s system 99% cpu 3.128 total
    rg erzg4 k.json > /dev/null  1.76s user 1.41s system 97% cpu 3.265 total

    # rg --no-mmap
    zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json --no-mmap >/dev/null); done
    rg erzg4 k.json --no-mmap > /dev/null  0.98s user 0.75s system 99% cpu 1.743 total
    rg erzg4 k.json --no-mmap > /dev/null  0.99s user 0.75s system 99% cpu 1.740 total
    rg erzg4 k.json --no-mmap > /dev/null  1.01s user 0.76s system 99% cpu 1.772 total
    rg erzg4 k.json --no-mmap > /dev/null  0.99s user 0.75s system 99% cpu 1.754 total

    # rg -j4
    zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 >/dev/null); done
    rg erzg4 k.json -j4 > /dev/null  1.75s user 1.35s system 98% cpu 3.134 total
    rg erzg4 k.json -j4 > /dev/null  1.75s user 1.44s system 98% cpu 3.224 total
    rg erzg4 k.json -j4 > /dev/null  1.80s user 1.38s system 99% cpu 3.204 total
    rg erzg4 k.json -j4 > /dev/null  1.80s user 1.35s system 99% cpu 3.164 total

    # rg -j4 --no-mmap
    zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 --no-mmap >/dev/null); done
    rg erzg4 k.json -j4 --no-mmap > /dev/null  0.98s user 0.75s system 99% cpu 1.740 total
    rg erzg4 k.json -j4 --no-mmap > /dev/null  0.97s user 0.74s system 99% cpu 1.721 total
    rg erzg4 k.json -j4 --no-mmap > /dev/null  0.99s user 0.75s system 99% cpu 1.752 total
    rg erzg4 k.json -j4 --no-mmap > /dev/null  0.98s user 0.76s system 99% cpu 1.748 total

Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac.

burntsushi 9 years ago

Wow. Those are awesome results, thank you.
> Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac.
I will fix that in ripgrep proper by making --no-mmap the default on Mac. :-) It should be an easy one to knock off: https://github.com/BurntSushi/ripgrep/issues/36
> not my first rodeo
Right, sorry about that. :-) Just had to cover all my bases!
(Also, `-j` on a single file won't do anything, and ripgrep should try to use multiple threads by default when searching multiple files.)
- lobster_johnson 9 years ago
  
  I suspected that -j wouldn't do anything on a single file. For three large (6.5GB in total) files I'm getting good performance, about 1.6x of what GNU Grep does, best case.