Also, maybe it's just me, but I feel like the sqlite-based thing is, if anything, kind of a downside. Use it for caching and calculation, sure, but I want the source of truth to be just plain markdown files I can take into other apps in 5 years when whatever I'm using now inevitably dies.
I know many note-taking users like the concept of markdown and plain text. So do I. However, I think plain text has its limitations. There are many other types of structured data in life. Just like Word and Excel, they have different responsibilities. No one worries that Excel will not work in a few decades because it's just an offline software. Similarly, no one worries that SQLite will be unable to open or view because it has already been running with billions of instances, becoming part of the infrastructure.
My general answer to this is to make sure whatever thing I'm building understands Pg COPY TSV syntax - specified by https://www.postgresql.org/docs/current/sql-copy.html under 'text format' - and export to/import from that.
It's nice to have something that plays nicely with awk/cut/diff/etc. and can be committed into a git (or elsevcs) repo.
(I'm not specifically wed to COPY as the form, but given it's well known and documented I've found it a good default)
What logseq is going to do is just dump documents to markdown (right now they use markdown),I think something along these lines is the way to go, store in sqlite and have a constant background process creating markdown files for everything
good idea. all documents are currently stored in the eidos__docs table. The `content` field is used to store the state of lexical documents in JSON format. Additionally, there is also a `markdown` field. This can be viewed by any sqlite software. Every time a document is updated, both fields are updated simultaneously, making it easy to convert to a file.
SQLite won't stop working but your format is proprietary. A markdown can be understood even without rendering to HTML but your tables are useless for an user without your app.
> SQLite won't stop working but your format is proprietary.
As far as I can tell, the whole app is open source (AGPL), and sqlite is obviously open source as well. I agree that markdown is more portable, but this setup is far from proprietary.
I may not have made the same decision (I love having my notes in markdown), but I think OP was quite thoughtful about it and has made an informed and reasonable decision.
You can likewise encode proprietary information in Markdown, in the sense that it can be readable but not necessarily understandable by another application that understands Markdown. Like say for example my application allows for tables that include formulas in cells. Perfectly human-readable Markdown but probably won’t have the same level of fidelity in another application.
A key idea of Eidos is to make each table a real SQLite table, so users can view and modify it through other software or visualize it with tools like Metabase
This is why I use RTF. It is considerably more flexible than Markdown (which can't do basic stuff like making text red), but not a proprietary format. The application I use (DEVONthink) supports a number of formats including Markdown, HTML and RTF, and documents how to get at the raw files if the application ever becomes unavailable. No use of a proprietary database format. This is for my 30-year files.
I also use RTFD in the same application. This is a variant of RTF which can have embedded images. It's not universally supported like RTF, but there are sufficient third-party editors to leave me confident that I will be able to get at the information. This is for my 5-year files.
I’ve been keeping an eye on these open source block based apps, and this kind of feels like you set out to scratch your itch and managed to release the most feature complete and polished one out there. Well done!
Something about SQLite at the core just seems to drive a direction I like, and pushes an open ethos. (Shout outs to Grist in this arena as well.) So much power in such simple interfaces… and when I saw you could just drop in your own SQL into Eidos, I shed a tear at the graveyard of my never-quite-working attempts at data-fying Obsidian.
As a long-time dilettante in this space, I totally agree. I am all-in on Obsidian (and a long-term paying sync customer) which is imo peerless for note-taking, and of course the plugin ecosystem is very vibrant.
But there’s no question that there’s something special about the core “block” abstraction used by Notion (and, I see, AnyType), in that it really does allow you to not only structure and display information in absolutely any way you want, but also to reason about that structure in a unified way. You can come back after a month away and still remember how everything fits together. You can literally build everything a team needs: wikis, datatables, highly customised task/project managers, CRM, etc. And it’s all multiplayer.
Notion the _product_ ain’t it, though. Closed-source, SaaS, very slow, buggy electron apps (Obsidian’s electron apps are very good, by the by - and their sync service is faultless). Hopeless handling of copy-pasted markdown: why can’t I simply paste markdown content in from another app and have it appear - and be editable - as a normal page? Insufficient access controls, no locking of page designs, poor sharing UX (just let me copy a link to the page without throwing up an access permissions dialog box each time and warning me that certain people won’t have access when they definitely will).
The final complaint would be that they’re sort of caught between different audiences. We already had a company wiki we built with the block abstraction and were very happy with, then they went and introduced a new “Wiki” … I dunno … “special type”. Is it composed just of blocks like everything else, or does it get some kind of special treatment. Is every page I create under the “Wiki” folder a normal page, or somehow special? I get that they have to walk a line between power and casual users, but any UX decision that obscures the core block abstraction fails to strike that balance, imo. Why didn’t they make Wikis just another official template built of normal blocks?
All of which is to say: I remain really hopeful on this space and would love to see an open source project come along and take the crown. Certainly an increasing number of promising entrants.
You maybe want to check the trademarks for this name, there are some for it. Not that this must become a problem, but it's better to avoid potential problems early.
Besides that, SQLite seems promising, but not sure how exactly this relates to everything. The website is talking about saving in browser, and there is no release on the GitHub. Is this something running in browser only?
Thanks for the reminder. I know this makes SEO more challenging, and it also reminds people of a certain game company, but I really love this concept, which originates from the Plato's theory of forms, and it pairs perfectly with the eidos.space domain name. While it may not be a great business name, the concept itself is exciting.
Eidos is deployed on Cloudflare Pages. it's a purely frontend project and the web is only used for distribution. In order to iterate faster, I didn't use GitHub's release yet.
There are some enhanced experience services running on serverless, but they are not necessary. For instance, adding a bookmark block to a document requires fetching extra information through a URL. This relies on https://github.com/mayneyao/link-preview . Most of the CORS related work is implemented through Cloudflare worker. These services are not necessary. Moreover, they are open-sourced and can be self-deployed. For more details, see https://github.com/mayneyao/eidos/blob/main/lib/const.ts#L64
You can use Eidos with Wi-Fi off. when you add a bookmark, it won't display a rich information card, but just a URL. if you don't like this part, I can provide options to disable them. All of this can be solved through Electron, but I think PWA is enough, and Cloudflare generously provides some services.
I have tried quite many such apps and keep returning to Tiddlywiki (https://tiddlywiki.com/). It is not perfect, and the lack of hierarchy can be both a blessing and a curse. It uses flat-files which can impact performance and be more cumbersome than a database. Also, the integration with external files is a bit clumsy.
However, the main strength is customizability. Various data is best presented in various ways, and separating data/content and presentation/template/layout while keeping them tightly integrated is incredibly powerful.
Cudos for thinking long-term with SQLite, avoiding lock-in is crucial for these kinds of apps!
* Tiddlywiki has existed for many years (first version came 20 years ago, the rewritten version left beta 10 years ago), and longevity and backward compatibility are key values for the developers
* A single-file Tiddlywiki is a single file that only requires a web browser to be used (e.g., you can email it to someone and they can just open it)
* Tiddlywiki is extremely customizable, much more than obsidian. It's almost hard to describe, but you can make very interesting things with just basic knowledge about html and css.
To cite myself:
> the problem with TiddlyWiki is that it's hard to describe since its so flexible. I have a few local tiddlywikis that I use for catalogues and note taking/knowledge base. The note taking was inspired by another HN comment [3] that introduced Drift [4,5], which includes features such as tabs for backlinks, keywords, and freelinks. But you may also find the Projectify edition [6] interesting or one cloning Roam Research [7].
Joe Armstrong of Erlang fame was a big fan of tiddlywiki, and he met up with the creator to give a talk together about tiddlywiki: https://youtube.com/watch?v=Uv1UfLPK7_Q
“Joe Armstrong & Jeremy Ruston - Intertwingling the Tiddlywiki with Erlang | Code Mesh LDN 18”
Yes, but the tiddlywiki file is the entire wiki. You've perhaps tried the dataview plugin for obsidian? Tiddlywiki is built on a similar idea, but much more flexible and powerful. However, tiddlywiki doesn't support in-text definition of fields (as far as I know)
I am a big fan of Notion but lately it's like they have been asleep on the driver seat. If they only offered custom extensions I think they would do so much better. I have recently switched to Obsidian but I am not fully committed yet, so i will give this a try.
My only critic so far... It seems you copied one of my least favorite "features" from Notion which is to force a default "Title" column for the tables. I find that most of the time I don't need a Title column, but I can't turn it off nor can you change it's type.
Wait a second. Do you mean there is no sync possibility? Because the browser storage is far from reliable. I don't see this as "safe" if the data is not synced or backed up somewhere else.
You can make some adjustments in the settings[1] to store data in a local folder. Then, use iCloud, Git, or your preferred service to back up your data. Just like the web version of VSCode[2], it can handle local files. The web is just an app and doesn't hold any data.
I have been making notes for a decade now. I have moved through plain txt files, no extension (linux style) txt files, CherryTree, SimpleNotes, LogSeq, Notion, GitHub Pages, Obsidian and may be more.
I have moved through phases of putting notes in physical notebook and scanning. I have 10s of scanned notebooks.
I have taken digital notes via stylus, via wacom pen and tablet, Samsung notes (best note taking app with stylus), a bunch of linux apps (xournal, xournal++ etc) and more...
I have written notes in pure html.
I have written personal app for taking notes in browser and storing it in postgres.
There is nothing better than text + git. Markdown is second.
My current configuration is: text/markdown + git + vs code + (plain) obsidian + plantuml + mermaid + local-git-server/github/gitlab etc.
I really wanted to like Notion but it's not smooth enough for writing. The cell system makes it clumsy.
I prefer to write on markdown files because it's much faster and I can do it on my text editor of choice. I like obsidian because it's basically that with a bit of extras.
But then I lose the concurrent editing.
I want Google Docs meets obsidian type of environment. And I am yet to find it.
I want this too. I add one thing: I want the ability to search/read/add to notes on both desktop and on mobile in a native app. I have yet to see a PWA that has the fluidity, stability, and ease of use of a well made app.
Congrats on the effort. Looking forward to have this expanded or further developed. The reason why I still use Notion is definitely for their database feature and an usable mobile app. It’s hard coming back to desktop-only
This looks cool. At first pass, I have one big question: are there any facilities or even just plumbing for syncing between installs / front-ends?
SaaS/cloud approaches have real downsides in terms of ownership but the big upside is access from multiple points/devices (home desktop, browser on office machine, personal laptop, mobile device, etc). Obviously SaaS isn't the only way to do that, centralizing around self-hosting or some syncing feature can work too. Anything like that on the radar?
I carefully designed the architecture, which is very flexible. The "backend" is currently running on web worker & service worker. It should be possible to deploy it to a web-standard runtime environment for self-hosting.
P2P synchronization based on CRDT is on the roadmap.
This post complains about Notion being slow, but when I try out the demo myself Eidos is slow as molasses. Forms are populated full seconds after I write in them, and I accidentally created several tables on accident because they again took multiple seconds to exist.
Very nice work Do you mind elaborating on how this differs from “outline” and “anytype”? I think a comparison table on the website on how this solution differentiates itself from competitors is going to be helpful. Also, what’s going to be the pricing model?
Thanks for your advice. Eidos is more like a PDM tool rather than a PKM tool, but it can be used as a PKM with some extensions, just as the introduction says, it's an extensible framework.
I've used the tools you mentioned and more. None of them have a powerful table, which is one of the reasons I decided to create Eidos. Eidos can handle large amounts of data like Excel. Anytype/Outline cannot do this.
I plan to follow Obsidian's model, with the core being always free. here is some plans:
- publish service (subscription-based)
- peer-to-peer data synchronization (One-time payment, like Resilio sync)
- an extension store that benefits developers (still a rough idea in my mind)
If you allow dependency-included self-host (preferably via an application wrapper like Electron where I can use a firewall to control which outbound connections are allowed), support mobile devices, and charge a one-time fee for p2p sync (decentralized, like SyncThing), this will be an instant buy for me.
Thought: Can you add a logout button?
Also, please consider a name change so SquEnix doesn't sue you out of existence!
What you want is also what I'm working on. Eidos is a personal open-source project focused on individual needs. It respects everyone's privacy. It does not collect any information. Since there is no account system, no need for a logout button
I really love this name. If I use Eidos Space, will there be trademark issues?
1. How to use extensions? I can create one (it looks like a javascript function getting some context and one argument) but I have no idea how to invoke it. I expected to be able to invoke them on selected text.
2. Does it sync on other computers / browsers or is it only local?
Sorry for lacking documents and guidance. When you enable a script, you can trigger it by typing 「/」 in command palette （cmd + k）. Currently, the selected text only works with the prompt. You reminded me that the script should also be able to handle selected text, and I will add this feature.
> Users that only compile trusted code are not impacted.
Doesn't Babel typically only compile your source code, and not what's in node_modules? I guess it depends if you're compiling single files, leaving import statements as they are; or creating a single bundle for the browser that includes external dependencies. For the latter, I imagine there's a chance some malicious package could exploit this.
How do you deal with iOS deleting PWA data when unused? I'm building an app that relies on indexedDB, which afaik is the only persistent storage a PWA can access.
Could one use this to say host a public docs site or blog? Does it have a backend server that needs to run or can I just upload a folder to S3 and have it work as a static site?
- Eidos has an Airtable-like table, but Obsidian does not.
- Eidos and Obsidian are both file-based.
- Eidos is based on SQLite, and Obsidian is based on Markdown.
I developed an extension to implement the import function, it's still in development and lacking documentation. You can try it out, but I'm not sure if there are any bugs. Feel free to join our Discord to stay updated.
Yes, much like the shortcomings of Windows 95 that spawned a million Notepad clones in the form of IDEs, code-oriented editors, and Office (née Works?)-like suites.
[1]: https://en.m.wikipedia.org/wiki/Eidos_Interactive
I think their Eidos-Labs research department is still active.
Personally, starting a new app called Eidos feels like a headache in the making.
Also, maybe it's just me, but I feel like the sqlite-based thing is, if anything, kind of a downside. Use it for caching and calculation, sure, but I want the source of truth to be just plain markdown files I can take into other apps in 5 years when whatever I'm using now inevitably dies.
It's nice to have something that plays nicely with awk/cut/diff/etc. and can be committed into a git (or elsevcs) repo.
(I'm not specifically wed to COPY as the form, but given it's well known and documented I've found it a good default)
As far as I can tell, the whole app is open source (AGPL), and sqlite is obviously open source as well. I agree that markdown is more portable, but this setup is far from proprietary.
I may not have made the same decision (I love having my notes in markdown), but I think OP was quite thoughtful about it and has made an informed and reasonable decision.
https://eidos.space/show/sqlite.webp
I also use RTFD in the same application. This is a variant of RTF which can have embedded images. It's not universally supported like RTF, but there are sufficient third-party editors to leave me confident that I will be able to get at the information. This is for my 5-year files.
Something about SQLite at the core just seems to drive a direction I like, and pushes an open ethos. (Shout outs to Grist in this arena as well.) So much power in such simple interfaces… and when I saw you could just drop in your own SQL into Eidos, I shed a tear at the graveyard of my never-quite-working attempts at data-fying Obsidian.
But there’s no question that there’s something special about the core “block” abstraction used by Notion (and, I see, AnyType), in that it really does allow you to not only structure and display information in absolutely any way you want, but also to reason about that structure in a unified way. You can come back after a month away and still remember how everything fits together. You can literally build everything a team needs: wikis, datatables, highly customised task/project managers, CRM, etc. And it’s all multiplayer.
Notion the _product_ ain’t it, though. Closed-source, SaaS, very slow, buggy electron apps (Obsidian’s electron apps are very good, by the by - and their sync service is faultless). Hopeless handling of copy-pasted markdown: why can’t I simply paste markdown content in from another app and have it appear - and be editable - as a normal page? Insufficient access controls, no locking of page designs, poor sharing UX (just let me copy a link to the page without throwing up an access permissions dialog box each time and warning me that certain people won’t have access when they definitely will).
The final complaint would be that they’re sort of caught between different audiences. We already had a company wiki we built with the block abstraction and were very happy with, then they went and introduced a new “Wiki” … I dunno … “special type”. Is it composed just of blocks like everything else, or does it get some kind of special treatment. Is every page I create under the “Wiki” folder a normal page, or somehow special? I get that they have to walk a line between power and casual users, but any UX decision that obscures the core block abstraction fails to strike that balance, imo. Why didn’t they make Wikis just another official template built of normal blocks?
All of which is to say: I remain really hopeful on this space and would love to see an open source project come along and take the crown. Certainly an increasing number of promising entrants.
I was coincidentally looking at Anytype yesterday, but can't create my own extensions (and the database has no formula type for columns??).
Curious if there's something that's been around for 1 year+ and allows me to build my own extensions?
What is not working for you if I may ask?
Besides that, SQLite seems promising, but not sure how exactly this relates to everything. The website is talking about saving in browser, and there is no release on the GitHub. Is this something running in browser only?
Eidos is deployed on Cloudflare Pages. it's a purely frontend project and the web is only used for distribution. In order to iterate faster, I didn't use GitHub's release yet.
There are some enhanced experience services running on serverless, but they are not necessary. For instance, adding a bookmark block to a document requires fetching extra information through a URL. This relies on https://github.com/mayneyao/link-preview . Most of the CORS related work is implemented through Cloudflare worker. These services are not necessary. Moreover, they are open-sourced and can be self-deployed. For more details, see https://github.com/mayneyao/eidos/blob/main/lib/const.ts#L64
However, the main strength is customizability. Various data is best presented in various ways, and separating data/content and presentation/template/layout while keeping them tightly integrated is incredibly powerful.
Cudos for thinking long-term with SQLite, avoiding lock-in is crucial for these kinds of apps!
[1] https://tiddlywiki.com/
[2] https://syncthing.net/
[3] https://www.wireguard.com/
* A single-file Tiddlywiki is a single file that only requires a web browser to be used (e.g., you can email it to someone and they can just open it)
* Tiddlywiki is extremely customizable, much more than obsidian. It's almost hard to describe, but you can make very interesting things with just basic knowledge about html and css.
To cite myself:
> the problem with TiddlyWiki is that it's hard to describe since its so flexible. I have a few local tiddlywikis that I use for catalogues and note taking/knowledge base. The note taking was inspired by another HN comment [3] that introduced Drift [4,5], which includes features such as tabs for backlinks, keywords, and freelinks. But you may also find the Projectify edition [6] interesting or one cloning Roam Research [7].
[3] https://news.ycombinator.com/item?id=25305527
[4] Source: https://github.com/bmann/drift-tiddlywiki-template/tree/mast...
[5] Demo: https://ramirosalas.com/
[6] https://thaddeusjiang.github.io/Projectify/
[7] https://rr-tw5.github.io/
“Joe Armstrong & Jeremy Ruston - Intertwingling the Tiddlywiki with Erlang | Code Mesh LDN 18”
Rather different from a closed-source desktop file editor.
My only critic so far... It seems you copied one of my least favorite "features" from Notion which is to force a default "Title" column for the tables. I find that most of the time I don't need a Title column, but I can't turn it off nor can you change it's type.
1. https://eidos.space/settings/storage 2. https://developer.chrome.com/blog/persistent-permissions-for...
I have moved through phases of putting notes in physical notebook and scanning. I have 10s of scanned notebooks.
I have taken digital notes via stylus, via wacom pen and tablet, Samsung notes (best note taking app with stylus), a bunch of linux apps (xournal, xournal++ etc) and more...
I have written notes in pure html.
I have written personal app for taking notes in browser and storing it in postgres.
There is nothing better than text + git. Markdown is second.
My current configuration is: text/markdown + git + vs code + (plain) obsidian + plantuml + mermaid + local-git-server/github/gitlab etc.
I really wanted to like Notion but it's not smooth enough for writing. The cell system makes it clumsy.
I prefer to write on markdown files because it's much faster and I can do it on my text editor of choice. I like obsidian because it's basically that with a bit of extras.
But then I lose the concurrent editing.
I want Google Docs meets obsidian type of environment. And I am yet to find it.
If any of you know of one, please let me know!
But if you want collaborative markdown in general, there are solutions like Hack.md
Something like cr-sqlite could potentially fix that though, although I'd prefer just plain markdown files for the backend.
SaaS/cloud approaches have real downsides in terms of ownership but the big upside is access from multiple points/devices (home desktop, browser on office machine, personal laptop, mobile device, etc). Obviously SaaS isn't the only way to do that, centralizing around self-hosting or some syncing feature can work too. Anything like that on the radar?
P2P synchronization based on CRDT is on the roadmap.
I've used the tools you mentioned and more. None of them have a powerful table, which is one of the reasons I decided to create Eidos. Eidos can handle large amounts of data like Excel. Anytype/Outline cannot do this.
I plan to follow Obsidian's model, with the core being always free. here is some plans: - publish service (subscription-based) - peer-to-peer data synchronization (One-time payment, like Resilio sync) - an extension store that benefits developers (still a rough idea in my mind)
Thought: Can you add a logout button?
Also, please consider a name change so SquEnix doesn't sue you out of existence!
I really love this name. If I use Eidos Space, will there be trademark issues?
1. How to use extensions? I can create one (it looks like a javascript function getting some context and one argument) but I have no idea how to invoke it. I expected to be able to invoke them on selected text.
2. Does it sync on other computers / browsers or is it only local?
Sync is on the roadmap.
Please make it no-code friendly for one-click easy installation if possible ( Similar to how WordPress or Drupal can be installed )
@babel/traverse <7.23.2
https://github.com/advisories/GHSA-67hx-6x53-jw92
Doesn't Babel typically only compile your source code, and not what's in node_modules? I guess it depends if you're compiling single files, leaving import statements as they are; or creating a single bundle for the browser that includes external dependencies. For the latter, I imagine there's a chance some malicious package could exploit this.
Also when it comes to LLM stuff, I would like to hook up AI's via a OpenAI Compatible LLM such as LiteLLM and Ollama
https://github.com/mayneyao/eidos/issues/107
:)
https://github.com/mayneyao/eidos-extension-importer
[0] https://tiddlywiki.com/
Contrary to your belief there are still people out there living outside the apple bubble.