nonethewiser 2 years ago

It seems “Codepilot” is being used as a general name for a coding focused language model. A more literal interpretation of the title suggests you can host Microsoft’s model yourself, which does not seem to be the case (unsurprisingly).

I guess I’m just surprised to see it used this way but “coding focused language model” doesn’t exactly roll off the tongue.

  • circuit10 2 years ago

    It's called that because it's a implementation of the Copilot API

  • valine 2 years ago

    I always assumed it a name for the application of a language model, ie I’m using this language model as a programming copilot. Some models making better copilots than others.

  • RobotToaster 2 years ago

    Could always take the facebook approach and call them CoFLaM.

MayeulC 2 years ago

Promising. Combine that with llama or BLOOM, perhaps finetuned on code, and perhaps your own codebase/documentation, and you cold have an interesting solution.

  • jncraton 2 years ago

    What is wrong with the CodeGen model that they are using?

    It is a reasonably large model (up to 16B params) that has already been trained on both natural language and code. I would expect it to underperform larger models, including GPT-3.5 and GPT-4, but this should still be very useful for autocomplete and boilerplate in simpler cases. It is a bit under trained compared to Chinchilla, T5, or LLaMA, but it still performs well.

    According to the paper[1], this model is competitive with the largest Codex model that was the basis for Copilot originally.

    [1] https://arxiv.org/pdf/2203.13474.pdf

    • MayeulC 2 years ago

      I haven't ran any of these models yet, I had just assumed CodeGen was less performant for "understanding" prompts. You are right that it's probably enough, especially if fine-tuning is an option.

      Now, I wonder: as the code-base grows, how often, and how, should such tuning take place?

qwerty3344 2 years ago

Would love to see a side by side comparison of the completions from this with completions from Copilot

  • make3 2 years ago

    this is likely much worse, if copilot uses gpt3.5 turbo or gpt4. who knows though what they're actually using.

    • mirthflat83 2 years ago

      Copilot before today was based on GPT-3

      • SparkyMcUnicorn 2 years ago

        Copilot uses "OpenAI’s Codex model, a descendent of GPT-3".

        Soon to be GPT-4, though.

      • sebzim4500 2 years ago

        How is this possible? They don't even use the same tokens, right? Whitespace is different IIRC.

        • ImaCake 2 years ago

          I think the GPT-3 thing is wrong, but it is not impossible. You can always train an existing neural net to do new things right? Not sure how to handle different whitespace tokens.

    • winddude 2 years ago

      the release today for copilot X says it's using gpt4 "With chat and terminal interfaces, support for pull requests, and early adoption of OpenAI’s GPT-4" ~ https://github.com/features/preview/copilot-x

      • zarzavat 2 years ago

        They only positively say that they are using GPT-4 for the new pull requests feature (and one other feature that I forgot). It’s unclear what model they are using for the main copilot code generation feature. It may be that they are only using GPT-4 for features that require the larger context size.

      • 5n 2 years ago

        I'm pretty sure that the actual Copilot auto completions are still on GPT3 for speed and cost reasons

        • winddude 2 years ago

          Yes, wait list for access to copilotX

anotheryou 2 years ago

I'd also be interested in a "bring your own OpenAI API key" Copilot X clone if anyone knows of one :)

  • plaguuuuuu 2 years ago

    I feel like that's going to be way more expensive than just using Copilot?

    Esp since copilot is updating to GPT4 soon anyway

    • jaimex2 2 years ago

      They cost about the same so you might as well have one.

    • anotheryou 2 years ago

      api is quite expensive, but I don't code daily. so no fixed cost is a big plus for me.

      (gpt4 api has no base costs like chatGPT plus)

phantom32 2 years ago

I wonder if FauxPilot's models (Salesforce Codegen family) can be quantized and run on the CPU. I was able to run the 350M model on my machine but it wasn't able to compete with Copilot in any way. Salesforce claims their model is competitive with OpenAI Codex their github description[1]. Maybe their largest 16B model is, but I haven't been able to try it.

[1] https://github.com/salesforce/CodeGen

toastal 2 years ago

Are there alternatives that don't require an Nvidia GPU and works with AMD? And more on-brand being hosted on a different code forge?

  • jaimex2 2 years ago

    Should be possible to make it work on AMD. CUDA is just easier to work with.

    • VadimPR 2 years ago

      This makes me glad to have chosen an nvidia gpu on linux. You get a whole lot of hidden benefits / QoL improvements that just aren't there with amd and nobody really mentions.

      • toastal 2 years ago

        You also have to deal with proprietary drivers. This can be mostly alright until it's not. Look at the hidden costs of dealing with Nvidia for Wayland for instance. CUDA lock-in is like DirectX lock-in. It should be preferable to have something platform agnostic, not just to support the current alternatives, but for new players to be able to join by supporting the common, open APIs as well.

siva7 2 years ago

Careful - If Microsoft doesn't like your wording (open-source GitHub Copilot server ) you'll be quickly kicked off Github

__anon-2023__ 2 years ago

From the setup.sh, the VRAM requirements are:

    echo "[1] codegen-350M-mono (2GB total VRAM required; Python-only)"
    echo "[2] codegen-350M-multi (2GB total VRAM required; multi-language)"
    echo "[3] codegen-2B-mono (7GB total VRAM required; Python-only)"
    echo "[4] codegen-2B-multi (7GB total VRAM required; multi-language)"
    echo "[5] codegen-6B-mono (13GB total VRAM required; Python-only)"
    echo "[6] codegen-6B-multi (13GB total VRAM required; multi-language)"
    echo "[7] codegen-16B-mono (32GB total VRAM required; Python-only)"
    echo "[8] codegen-16B-multi (32GB total VRAM required; multi-language)"
So I could try the 350M models on a laptop with an NVidia card of 2GB.

Another factoid, I noticed vultr is offering fauxpilot images in their GPU instance provisioning menu.

The model used (Salesforce Codegen) is on huggingface: https://huggingface.co/models?search=salesforce+codegen

syntaxing 2 years ago

Anyone try some quantized model with this like llama?

nu11ptr 2 years ago

I wonder... is MS likely going to make it harder to swap the URL on the official copilot client in light of this? Will they continue hosting the "competition"?

  • californical 2 years ago

    There’s an alternative client that already exists for neovim, it’s actually better than the official client IMO, and should be easy to change the URL there!

  • Takennickname 2 years ago

    I'm pretty sure VS Code Extensions aren't compiled.

    • benatkin 2 years ago

      They can still lock them down, like Mozilla did with Firefox extensions...

      • neximo64 2 years ago

        VS Code is an electron app so much harder

        • benatkin 2 years ago

          I don't think so, in both places have code that's running the extensions that you can change.

          • neximo64 2 years ago

            Except one of them is a C/Rust app, and one is a Javascript app. So one is harder to lock down extensions on than the other.

  • powersnail 2 years ago

    The integration with Copilot doesn’t quite compete with MS, because, well, using the extension requires a functioning subscription. You’d be paying MS, regardless of whether you point it to fauxpilot or copilot.

    But there are other extensions that are specifically made for fauxpilot which work well.

  • circuit10 2 years ago

    It has existed since October 2022, so...

    • nu11ptr 2 years ago

      This is the first I'm hearing of it. I wonder how many others as well are seeing this via HN the first time? I would imagine MS has been aware of this since more or less inception, however, any product threat isn't really about existence, but how viable and how well known/used it is.

tommica 2 years ago

So much potential in this! Would love to run this locally!

_aaed 2 years ago

Anyone knows how this compares to Tabnine?

  • marcusbuffett 2 years ago

    Copilot is a million times better then Tabnine. Tabnine was promising, but totally stopped making any improvements to the model after it got bought out years ago.

  • stuaxo 2 years ago

    Is Tabnine still going (and is it any good?) I used to use Kite back in the day.

    • _aaed 2 years ago

      The free version is average at best. It only really autocompletes a line (if you're lucky). It usually autocompletes a few characters in an expression