-
@ mleku
2025-05-13 21:25:25
tried upping the number of worker threads and it blows out memory but giving the worker thread one buffer seems to keep memory under control... the full text index is very slow, it does a lot of writing to the value table, well, it's fine really, once it's done with the existing store of events it only updates events as they come in which will be fine, and it only has to do it once because events are immutable
i probably should think about the deleting as well, since a deleted event should really be deleting all of its fields in the inverted indexes, probably should even do that next... and it doesn't have to require a big amount of work, it can just gather the words in the event and go straight to them to prune out the entry, or in other words only a little more heavy than making the entries in the first place. has to be done because replaceable events delete as well as actual delete events. yeah, gotta attend to that next, before i continue to the search.