I’m working Telescope - an open-source web-based log viewer designed to make working with logs stored in ClickHouse easier and more intuitive.
I wasn’t happy with existing log viewers - most of them force a specific log format, are tied to ingestion pipelines, or are just a small part of a larger platform. Others didn’t display logs the way I wanted.
So I decided to build my own lightweight, flexible log viewer - one that actually fits my needs.
Check it out:
Video demo: https://www.youtube.com/watch?v=5IItMOXwugY
GitHub: https://github.com/iamtelescope/telescope
Live demo: https://telescope.humanuser.net
Discord: https://discord.gg/rXpjDnEc
This looks pretty cool, I love seeing more clickhouse-native logging platforms springing up! It's a surprisingly underrated platform to build on when I talk to other engineers.
I'm one of those authors of an existing log viewer (hyperdx) and was curious if we were one of those platforms that didn't fit your needs? Always love learning what use cases inspire different approaches.
Would this also work with something like Plausible (https://github.com/plausible/analytics) which uses ClickHouse to store web analytics data, or is it primarily for log data?
Despite the fact that Telescope is focused on application log data, it could be used for any type of data as long as it's stored in ClickHouse and has some time fields.
At the moment, I have no plans to support arbitrary data visualization in Telescope, as I believe there are better BI-like tools for that scenario.
Cool! I'm currently playing with the Grafana Clickhouse connector to do broadly similar - are these compatible? Can Telescope read an OTEL logs table in Clickhouse?
Yes, this is exactly where Telescope can be useful (and actually, the way Grafana displays logs was my motivation for writing my own viewer)
Telescope can work with any table in ClickHouse. Of course, not every single ClickHouse type has been tested, but there shouldn’t be any issues with the most common ones
Nice idea!
However, I’m not experienced enough with Vue (and frontend) development to properly design an exportable component. So, at least for now, I don’t think I’ll be able to make it happen myself.
It would be great if the logs could describe a bit what exactly one has to do to use this as an alternative to Grafana Loki.
How do I get my logs (e.g. local text files from disk like nginx logs, or files that need transformation like systemd journal logs) into ClickHouse in a way that's useful for Telescope?
What kind of indices do I have to configure so that queries are fast? Ideally with some examples.
How can I make that full-text substring search queries are fast (e.g. "unexpected error 123")? When I filter with regex, is that still fast / use indices?
From the docs it isn't quite clear to me how to configure the system so that I can just put a couple TB of logs into it and have queries be fast.
Telescope is primarily focused on log visualization, not on log collection or preparing ClickHouse for storage. The system does not currently provide (and I think will not ever) built-in mechanisms for ingesting logs from any sources.
I will consider providing a how-to guide on setting up log storage in ClickHouse, but I’m afraid I won’t be able to cover all possible scenarios. This is a highly specific topic that depends on the infrastructure and needs of each organization.
If you’re looking for a all-in-one solution that can*both collect and visualize logs, you might want to check out https://www.highlight.io or https://signoz.io or other similar projects.
And also, by the way, I’m not trying to create a "Grafana Loki killer" or a "killer" of any other tool. This is just an open source project - I simply want to build a great log viewer without worrying about how to attract users from Grafana Loki or Elastic or any other tool/product.
As someone who has never worked anywhere that tried it out, what do you not like about loki. I've been stuck in the very expensive splunk and opensearch/kibana mines for many years and I find it an amazingly frustrating place to be. I honestly find that I can better debug via logs using grep than either of those tools.
There's also Logdy (https://github.com/logdyhq/logdy-core) that can work with raw files and comes with a UI as well in a single precompiled binary so no need for installs and setups. If you're looking for a simple solution for browsing log files with a web UI, this might be it!
(I'm the author)
Heyo I’ve noticed Lodgy come up a few times on HN now, and was curious if you explored making it a proper desktop application instead of a two-part UI and CLI application. Did you rule that out for some reason?
If you're looking for this kind of UI also check out Coroot https://github.com/coroot/coroot which has awesome UI for logs and OpenTelemetry traces and also stores data in Clickhouse
I need a central place, something simple where I can actually read the contents of the logs that are generated by the dozen of services that I run for clients, etc… instead of stupidly SSH’ing to every server.
Does this fit the use case?
I tried Loki once but it was painful to set up and more geared toward aggregating events and stats.
Thanks! Telescope is more focused on displaying logs and providing access to them rather than handling log ingestion. In the future, I plan to support various sources like Docker, k8s, and files to improve the local development workflow. However, it's unlikely that Telescope will support fetching logs from remote servers via SSH, as that's not its primary use case.
If all you want is the plaintext logs, there's no need to bother with special products. Just point syslog in the right direction as if it was 1995. Everything can log to syslog already. Things like Splunk, Graylog and Kibana are mostly for visualization and query interfaces.
Only problematic thing might be relatively frequent storage changes (like they like to deprecate primary storage driver), otherwise its IMHO easy to setup. I'm running it on several projects, because it doesn't needs beefy machine like Elastic or even ClickHouse.
That sounds great! Do you have a link? I'd love to check it out.
For me, the most challenging parts are still ahead - live tailing and a plugin system to support different storage backends beyond just ClickHouse. Those will be interesting problems to solve! What was the biggest challenge for you?
Telescope is focused purely on viewing logs for existing data. It doesn’t enforce any specific ingestion setup or schema and doesn’t support traces or session storage.
You can think of it as just one part of a logging platform, where a full platform might consist of multiple components like a UI, ingestion engine, logging agent, and storage. In this setup, Telescope is only the UI.
I've used graylog the most so that's what it looks like to me :P. I like how you can do a bunch of extraction stuff right there in the query interface though, that's awesome. It seems like a very thoughtful UI.
Well, if you're happy with ELK, you should definitely use it! As I mentioned earlier, I’m not trying to sell anything or convince people to switch from their current solutions - just offering an alternative perspective on how things can be done.
My motto is "Know your data". I’m not a big fan of schemaless setups - I believe in designing a proper database schema rather than just pushing data into a black hole and hoping the system will handle it.
It is not same as OP, but according to a similar o11y stack on top of Clickhouse, Signoz, Clickhouse based logging costs less than ELK for storage and performs better:
We've found that ClickHouse is extremely fast for write-once/read-many. So its great for recording logs. If Telescope provides the search/index features that Elastic provides, this could be a nice performance bump. FWIW, I haven't tested Telescope, so this is all just my musing.
As we all know, naming is an unsolvable problem in IT :)
Regarding performance - 95% of Telescope's speed depends on how fast your ClickHouse responds. If you have a well-optimized schema and use the right indexes, Telescope's overhead will be minimal.
Rollbar has a feature to upload JavaScript sourcemaps files. When I am viewing logs from minified js files, it automatically apply sourcemaps and correctly shows line number.
Looks simple and clean! Big ups for starts of good screenshots, docs, and quickstart (Docker) instructions.
Regarding the name, "Telescope" is also the name of a Neovim fuzzy finder[0] that dominates the ecosystem there. Other results appear by searching "telescope github".
This one seems to be optimized for log viewing at the moment, are there any DataDog alternatives built on top of Clickhouse, which supports full range of OpenTelemetry features?
Hey everyone!
I’m working Telescope - an open-source web-based log viewer designed to make working with logs stored in ClickHouse easier and more intuitive.
I wasn’t happy with existing log viewers - most of them force a specific log format, are tied to ingestion pipelines, or are just a small part of a larger platform. Others didn’t display logs the way I wanted.
So I decided to build my own lightweight, flexible log viewer - one that actually fits my needs.
Check it out:
This looks pretty cool, I love seeing more clickhouse-native logging platforms springing up! It's a surprisingly underrated platform to build on when I talk to other engineers.
I'm one of those authors of an existing log viewer (hyperdx) and was curious if we were one of those platforms that didn't fit your needs? Always love learning what use cases inspire different approaches.
Would this also work with something like Plausible (https://github.com/plausible/analytics) which uses ClickHouse to store web analytics data, or is it primarily for log data?
Despite the fact that Telescope is focused on application log data, it could be used for any type of data as long as it's stored in ClickHouse and has some time fields.
At the moment, I have no plans to support arbitrary data visualization in Telescope, as I believe there are better BI-like tools for that scenario.
Yeah that's fair, thank you.
Very cool! Just digging in. Does it works with the new JSON format clickhouse introduced recently?
Also, what service did you use to make the video, if you don't mind my asking?
Thanks!
I haven't tested the new JSON format in ClickHouse yet, but even if something doesn't work at the moment, fixing it should be trivial.
As for the video service, it wasn’t actually a service but rather a set of local tools:
- Video capture/screenshots - macOS default tools
- Screenshot editing - GIMP
- Voice generation - https://elevenlabs.io/app/speech-synthesis/text-to-speech
- Montage – DaVinci Resolve 19
Cool! I'm currently playing with the Grafana Clickhouse connector to do broadly similar - are these compatible? Can Telescope read an OTEL logs table in Clickhouse?
Yes, this is exactly where Telescope can be useful (and actually, the way Grafana displays logs was my motivation for writing my own viewer)
Telescope can work with any table in ClickHouse. Of course, not every single ClickHouse type has been tested, but there shouldn’t be any issues with the most common ones
If you want, you can check how it works with the OTEL schema in the live demo here: https://telescope.humanuser.net/sources/otel-demo/explore
Very cool! Would be nice to have a library for the frontend components for the log viewer, to be able to reuse them in other projects :)
Nice idea! However, I’m not experienced enough with Vue (and frontend) development to properly design an exportable component. So, at least for now, I don’t think I’ll be able to make it happen myself.
It would be great if the logs could describe a bit what exactly one has to do to use this as an alternative to Grafana Loki.
How do I get my logs (e.g. local text files from disk like nginx logs, or files that need transformation like systemd journal logs) into ClickHouse in a way that's useful for Telescope?
What kind of indices do I have to configure so that queries are fast? Ideally with some examples.
How can I make that full-text substring search queries are fast (e.g. "unexpected error 123")? When I filter with regex, is that still fast / use indices?
From the docs it isn't quite clear to me how to configure the system so that I can just put a couple TB of logs into it and have queries be fast.
Thanks!
Telescope is primarily focused on log visualization, not on log collection or preparing ClickHouse for storage. The system does not currently provide (and I think will not ever) built-in mechanisms for ingesting logs from any sources.
I will consider providing a how-to guide on setting up log storage in ClickHouse, but I’m afraid I won’t be able to cover all possible scenarios. This is a highly specific topic that depends on the infrastructure and needs of each organization.
If you’re looking for a all-in-one solution that can*both collect and visualize logs, you might want to check out https://www.highlight.io or https://signoz.io or other similar projects.
And also, by the way, I’m not trying to create a "Grafana Loki killer" or a "killer" of any other tool. This is just an open source project - I simply want to build a great log viewer without worrying about how to attract users from Grafana Loki or Elastic or any other tool/product.
As someone who has never worked anywhere that tried it out, what do you not like about loki. I've been stuck in the very expensive splunk and opensearch/kibana mines for many years and I find it an amazingly frustrating place to be. I honestly find that I can better debug via logs using grep than either of those tools.
There's also Logdy (https://github.com/logdyhq/logdy-core) that can work with raw files and comes with a UI as well in a single precompiled binary so no need for installs and setups. If you're looking for a simple solution for browsing log files with a web UI, this might be it! (I'm the author)
Heyo I’ve noticed Lodgy come up a few times on HN now, and was curious if you explored making it a proper desktop application instead of a two-part UI and CLI application. Did you rule that out for some reason?
Looks cool!
If you're looking for this kind of UI also check out Coroot https://github.com/coroot/coroot which has awesome UI for logs and OpenTelemetry traces and also stores data in Clickhouse
On the naming front telescope is already use for a log viewer https://laravel.com/docs/11.x/telescope
If I search telescope logs on google that’s the top result for me.
Looks cool I might try it out!
I need a central place, something simple where I can actually read the contents of the logs that are generated by the dozen of services that I run for clients, etc… instead of stupidly SSH’ing to every server.
Does this fit the use case?
I tried Loki once but it was painful to set up and more geared toward aggregating events and stats.
Thanks! Telescope is more focused on displaying logs and providing access to them rather than handling log ingestion. In the future, I plan to support various sources like Docker, k8s, and files to improve the local development workflow. However, it's unlikely that Telescope will support fetching logs from remote servers via SSH, as that's not its primary use case.
If all you want is the plaintext logs, there's no need to bother with special products. Just point syslog in the right direction as if it was 1995. Everything can log to syslog already. Things like Splunk, Graylog and Kibana are mostly for visualization and query interfaces.
I'm author of Logdy: https://logdy.dev/ https://github.com/logdyhq/logdy-core It comes as a precompiled binary you can download/deploy on the server and use to browse larger log files. I suggest you take a look!
Graylog is a pretty standard solution to your problems (I believe), although they've been closing down their licensing more and more as time goes on.
I’m curious to know what makes the Loki installation process so painful.
I’m interested in learning more about the software installation experience.
Only problematic thing might be relatively frequent storage changes (like they like to deprecate primary storage driver), otherwise its IMHO easy to setup. I'm running it on several projects, because it doesn't needs beefy machine like Elastic or even ClickHouse.
I'd recommend VictoriaLogs and shipping to via Vector
genuinely wondering if https://multiplayer.app would work for you.
note: I'm part of the Multiplayer team.
Awesome stuff! Just published something similar today
Just curious, what is the most challenging thing in your opinion when building such log viewer?
That sounds great! Do you have a link? I'd love to check it out.
For me, the most challenging parts are still ahead - live tailing and a plugin system to support different storage backends beyond just ClickHouse. Those will be interesting problems to solve! What was the biggest challenge for you?
How is it different from Signoz, a complete observability stack (including Logs) built on top of Clickhouse?
Telescope is focused purely on viewing logs for existing data. It doesn’t enforce any specific ingestion setup or schema and doesn’t support traces or session storage.
You can think of it as just one part of a logging platform, where a full platform might consist of multiple components like a UI, ingestion engine, logging agent, and storage. In this setup, Telescope is only the UI.
I like how this is mostly based on the Kibana UI. Makes easier to convince other people to move to it.
To be honest, I was more inspired by DataDog :)
I've used graylog the most so that's what it looks like to me :P. I like how you can do a bunch of extraction stuff right there in the query interface though, that's awesome. It seems like a very thoughtful UI.
Honestly that pushes me away from it. I find kibana to be a very frustrating experience.
(not op) curious what you find frustrating about it?
Just curious, as I'm in the market - why should I use this instead of the ELK stack?
Well, if you're happy with ELK, you should definitely use it! As I mentioned earlier, I’m not trying to sell anything or convince people to switch from their current solutions - just offering an alternative perspective on how things can be done.
From my perspective, a ClickHouse-based setup can be cheaper and possibly faster in certain conditions – here’s some comparison made by ClickHouse Inc. - https://clickhouse.com/blog/clickhouse_vs_elasticsearch_the_...
My motto is "Know your data". I’m not a big fan of schemaless setups - I believe in designing a proper database schema rather than just pushing data into a black hole and hoping the system will handle it.
It is not same as OP, but according to a similar o11y stack on top of Clickhouse, Signoz, Clickhouse based logging costs less than ELK for storage and performs better:
https://github.com/SigNoz/logs-benchmark
We've found that ClickHouse is extremely fast for write-once/read-many. So its great for recording logs. If Telescope provides the search/index features that Elastic provides, this could be a nice performance bump. FWIW, I haven't tested Telescope, so this is all just my musing.
Look out, Kibana, they're gunning for you!
Unfortunate name choice, as @csh602 mentioned
Viewer looks pretty good though. Reminds me of DataDog UI, but not as slow. Will play around more, thanks!
As we all know, naming is an unsolvable problem in IT :)
Regarding performance - 95% of Telescope's speed depends on how fast your ClickHouse responds. If you have a well-optimized schema and use the right indexes, Telescope's overhead will be minimal.
It can display logs in-context. Awesome!
Rollbar has a feature to upload JavaScript sourcemaps files. When I am viewing logs from minified js files, it automatically apply sourcemaps and correctly shows line number.
Is there any open source tool that does the same?
Looks simple and clean! Big ups for starts of good screenshots, docs, and quickstart (Docker) instructions.
Regarding the name, "Telescope" is also the name of a Neovim fuzzy finder[0] that dominates the ecosystem there. Other results appear by searching "telescope github".
[0]: https://github.com/nvim-telescope/telescope.nvim
Clearly we need an extension to search this new service with telescope.nvim. telescope-telescope.nvim.
Also, a bit more directly related, a log viewer / monitoring solution from Laravel: https://laravel.com/docs/12.x/telescope
Well, every single name I came up with was already taken and present in GitHub. So...
This one seems to be optimized for log viewing at the moment, are there any DataDog alternatives built on top of Clickhouse, which supports full range of OpenTelemetry features?
There are these guys which are based on OTEL: https://github.com/hyperdxio/hyperdx
+1 for hyperdx, I'm testing it and love it.
author here, just wanted to chime in that I really appreciate the kind words about what we've been building :)
Check out signoz: https://github.com/SigNoz/signoz
OSS o11y platform built on clickhouse & otel.
There is lumigo, also based on Clickhouse https://lumigo.io/
I think https://multiplayer.app would also fit your description.
[dead]