Sarah Jamie Lewis

Cryptography and Privacy Researcher. President @ Open Privacy Research Society (@openprivacy).

Founder @ Blodeuwedd Labs (@blodeuweddlabs)

Building free and open source, privacy-enhancing, surveillance-resisting tech like Cwtch (@cwtch)

Public Key

npub14mfj9wu5ujvu6rxj8w62dvkvqj7myqc6kz0upz3kuw3wx4dz9vgstvp58n

NIP-05 Address

sarahjamielewis_at_mastodon.social@mostr.pub

Profile Code

nprofile1qqs2a5ezhw2wfxwdpnfrhd9xktxqf0djqvdtp87q3gmw8ghr2k3zkygpz4mhxue69uhhyetvv9ujumt0wd68ytnsw43qd8nd82

Publishing to

relay.mostr.pub

Author Public Key

npub14mfj9wu5ujvu6rxj8w62dvkvqj7myqc6kz0upz3kuw3wx4dz9vgstvp58n

Show more details

Last Notes

2024-11-17 05:51:21

- reply

The strongest argument I think I can conjure, is that I do rely quite a bit on terminal emulators while working on remote machines - I make heavy use of tmux and vim, and all the underlying cursed machinery that makes them work.
That being said, there are so many alternative workflows there both existant and imaginable that do not rely on the cursed machinery.

2024-09-17 18:51:03

Reading through Mozillas jobs page: "Senior Staff Machine Learning Engineer, Gen AI", "Senior Staff Software Engineer, Ads", "Program Manager, AI 360", "Principal Product Manager, Generative AI","Sr Staff Fullstack Engineer, Anonym", "Senior Manager, Sales (Anonym)"
I see they have fully pointed the ship towards a future of AI and Ads.

2024-09-11 18:12:31

I really try to like Firefox, but the last 5 minutes really captures the kind of papercut that happens often:
- I open a new tab and firefox informs me it has updated itself and needs to restart and won't allow any further operations until it does so.
- Fine, I close and restart.
- I reopen Firefox to find a brand new sponsored weather widget on my otherwise blank new tab page - from a source I would never otherwise visit.
Thanks for breaking my flow and the privacy breach, I guess.

2024-09-03 18:15:00

- reply

Don't you just love software.

2024-08-28 19:29:09

- reply

Pushed a few small updates, mostly fixing some pointer handling and properly supporting defined input regions.

2024-08-28 17:25:28

- reply

@npub1g0n…ffsr
Now at https://github.com/s-rah/nyctal :)

2024-08-28 17:24:55

- reply

I've listed a few tenets and a brief overview of the code and things that are likely to change in the near future in (https://github.com/s-rah/nyctal/blob/main/CONTRIBUTING.md)
If you are interested in working on an outstanding feature or bug fixes etc. Then feel free to reach out :)

2024-08-28 17:22:55

- reply

I built Nyctal because I like building operating systems as a hobby and have been wanting a small, easily maintainable display server for those projects; one that has the potential to be compatible with more mainstream applications.
To that end, apart from the desired functionalities outlined in the README, my goal is to keep Nyctal very small and easily understood by a single developer.

2024-08-28 17:19:36

- reply

If you'd like to try out Nyctal there are two options:
- nyctal-x11 - an example nyctal compositor running in an x11 window
- nyctal-dri - a no-safety compositor that accesses rendering and input devices directly (please only use this in qemu or something).
My plan is to eventually add a few more niceties (like libseat compatibility) so it can fit a little more naturally inside a modern linux system.

2024-08-28 17:10:44

Today I am open sourcing my tiny, zero dependency Wayland compositor Nyctal: https://github.com/s-rah/nyctal
(or about 3000 lines of code you would have to write anyway....)
Nyctal is far from "finished" - and you probably wouldn't want to deploy it. There is plenty of bugs to squash, and features/extensions I really want to add.
But it has reached a state where doing those things incrementally on a public code base is a better choice than to continue hacking away in private.

2024-08-25 20:35:53

- reply

(I assume there are very good reasons why configuring keyboard keymaps requires such eldritch schemes but I am very wisely choosing to not go down that path)

2024-08-25 17:59:47

Update on the tiny from-scratch zero-dep wayland compositor I've been building.
It can now handle keyboard input, xdg_popup surfaces, and a whole bunch of other niceties.
Which means that with LIBGL_ALWAYS_SOFTWARE=1 set it can now handle flutter apps (like cwtch) and some gnome apps (like calc) pretty nicely.
https://files.mastodon.social/media_attachments/files/113/023/944/093/126/052/original/2b4cc6b56541f803.mp4

2024-08-24 19:49:20

- reply

Also in my investigations have found a couple of wayland compositors not based on wlroots that don't seem to handle these faults - which means that any unprivileged client could crash the compositor.
Seems like the kind of thing that should be documented at the protocol level - pending a new linux nosigbus/nofault mmap flag

2024-08-24 19:37:11

Way too deep in wayland compositor internals now...but the realization that writing a safe/robust wayland compositor in go will *require* turning off page fault crashing in the language runtime is a fun one.
I understand why it must be this way, though it fills me with uneasy feelings.

2024-08-20 15:44:51

- reply

@npub13f0…2mvp
Thanks for the information. Looks like MozContainerWayland.cpp is failing to get a subsurface and ends up passing a null pointer into libwayland-client.
Guess I should submit this somewhere.

2024-08-19 20:45:45

Update on my mini from-scratch wayland compositor (previously: https://mastodon.social/@sarahjamielewis/112984114477365677)
The code now nicely handles clients coming and going, and I now have a "workspace" concept which can manage and layout windows (basic main-stack tiling) - including configuring them for resizing.
Also workspace now renders to video.
https://files.mastodon.social/media_attachments/files/112/990/619/802/097/947/original/d31c05efa5a688e3.mp4

2024-08-18 23:20:09

- reply

Some more tests:
- xfce4 terminal renders
-gnome calculator now renders (needed a couple more definitions on my side)
seems like most gtk apps will play nicely with my tiny little compositor.
https://files.mastodon.social/media_attachments/files/112/985/579/447/563/749/original/cf29ecdd37667f4c.jpeg

2024-08-18 21:34:18

- reply

Oh I've also tried a few electron apps with some magic use-wayland flags and they all enter a kind of infinite sync-loop - I assume also waiting for some additional extension to appear.

2024-08-18 21:32:41

- reply

if you know of other semi-mainstream apps with wayland support that might not be completely dependent on dmabuf then let me know - I'd like to get a broad idea of exactly what extensions are defacto-expected.

2024-08-18 21:30:09

- reply

The list of tests I have done so far on real world apps:
- zathura and thunar are both happy with my minimal compositor
- all flutter-based apps render a simple error window (no GL context)
- firefox actually segfaults, I don't understand why
- mpv rejects the compositor because of the lack of viewport extension support (at least it prints a nice error message)
- inkscape renders a simple error popup - also not sure exactly why.

2024-08-18 20:07:09

- reply

Managed to find at least one wayland app that can utilize shm for rendering - zathura
So my compositor can actually render a functional app!
https://files.mastodon.social/media_attachments/files/112/984/812/275/524/514/original/03307affee0e018e.jpeg

2024-08-18 17:06:10

For my weekend project I decided to write a wayland compositor from scratch (no deps, just feeling the UDS vibes and responding accordingly).
The very successful rendering of this crash indication window has me feeling many things.
https://files.mastodon.social/media_attachments/files/112/984/099/404/677/158/original/dd110fc6e8d86f84.jpeg

2024-08-17 22:08:26

Question: In the wayland protocol xml spec, the bind request is described as containing uint name, and a new_id id.
In practice, clients seem to universally provide the entire interface spec instead of the uint name - I've found one third party reference that indicates this happens because the version of such interfaces would be ambiguous.
My question is: where is this (actual) behaviour specified?

2024-08-17 03:24:16

- reply

None of those three options are time efficient - and If I am being honest with myself they are all firmly in the abyss, some are just more in the abyss than others.
I am also in this abyss.

2024-08-17 02:49:10

- reply

Even in a world where we conquer every abyss cycle; In what is likely a personal failing, I don't think I will every find happiness in a system critically dependent on multiple versions of perl, all autogenerating scripts that autogenerate more scripts that ultimately all produce the same 20 line makefile.

2024-08-17 02:43:04

- reply

I admire guix for the work they've done on full source bootstrapping, fixing problems like "to compile gnu grep you need gnu grep" and other such thorns.
Though, having recently spent time in a related abyss, I walk away with the feeling that undertaking all that work to keep those abyss cycles may be long-term detrimental to the tree.

2024-08-17 02:36:11

Once you account for the inherent complexity of build systems, the beautiful notion of a dependency "tree" shifts and mutates, ejecting you from the world of platonic forms and into the depths of cyclic abyss graphs.

2024-08-08 20:32:06

Had some time today to work on my little web / gemini browser.
I rewrote the rendering engine so it can now better handle lightweight html - including inline links. If its just headings, paragraphs, links and images then it should render not to badly.
https://files.mastodon.social/media_attachments/files/112/928/291/957/098/285/original/8c14542335b2c5b2.mp4

2024-07-23 17:26:03

- reply

Today's browser progress: some more UI niceties and inline images are now supported for feeds.
https://files.mastodon.social/media_attachments/files/112/836/965/277/718/315/original/6158a2166c942d19.mp4

2024-07-22 22:31:20

- reply

My little browser now supports subscribing to feeds, the most recent items can then be viewed in about:feeds
Prototyping custom application handlers.
https://files.mastodon.social/media_attachments/files/112/832/500/132/457/094/original/b397ab36c2223389.mp4

2024-07-22 19:43:35

- reply

This morning I split out the scheme and media type handling in my little browser, and wrote a stub https scheme handler, in addition to writing content type handlers for html, rss/atom and plain text.
So now my browser supports rendering rss feeds, and some bare bones html rendering for simple pages.
https://files.mastodon.social/media_attachments/files/112/831/841/637/406/505/original/5f7aae044b97af82.mp4

2024-07-21 20:27:19

- reply

Spent some time this morning writing a generic selection handler for my ui framework. Have it integrated with the browsing context so can now select all the text elements there with a mouse.
https://files.mastodon.social/media_attachments/files/112/826/353/688/783/010/original/dd663f60a800d409.mp4

2024-07-21 04:07:41

- reply

Pretty happy with the current architecture, I've deliberately tried to keep it as minimal as possible.
Next steps: refine the browser api so I can easily define new media type handlers / renderers. And add a few more UI toolkit niceties - especially around text handling (select / copy / paste etc.) .

2024-07-21 03:43:30

- reply

Added a few niceties like a back button, a refresh button and some page loading feedback
https://files.mastodon.social/media_attachments/files/112/822/408/193/699/042/original/c259339fd09d1478.mp4

2024-07-20 20:15:39

Started writing a little gemini browser.
Hand rolled UI framework, currently using MiniFB for windowing/mouse/keyboard support, everything else in Go.
https://files.mastodon.social/media_attachments/files/112/820/634/285/191/298/original/c61a3e9a2518db6c.mp4

2024-07-18 17:38:21

- reply

I'm now hosting a little experimental gemini server at gemini://resistant.tech/
All that is there right now are the details of the irc server I set up a few days ago (https://mastodon.social/@sarahjamielewis/112797393422934123) (feel free to drop by and say hi)
I will likely mirror my little microblog over there as well - and play around with some feed / interactive client ideas.

2024-07-18 17:20:03

- reply

I do feel it's a bit of a miss to have those per site identities, and not have a more complete PUT-story.
That hasn't stopped people experimenting with what interaction looks like over gemini - but that tension is definitely there.

2024-07-18 17:17:48

Following on from my last few threads about what the web should be, and some discussions in our little irc channel...
I spent yesterday evening deep diving into the gemini (protocol) ecosystem - wrote a few gemini servers, tried a few browsers.
Some thoughts:
I really like gemini, I like that writing a server takes a few minutes, I really like how small and uncomplicated the browsers are. Feels cozy.
I especially like the vision of supporting client side certificates / per-site identities.

2024-07-16 20:29:34

- reply

@npub1mr7…qlc2 I think mIRC is still a thing, and hexchat has windows builds I think.

2024-07-16 19:43:17

- reply

@npub1jtm…l72z I've heard good things about Revolution. Personally I use an ssh terminal and irssi.

2024-07-16 19:34:48

- reply

As a note: I am in meetings all afternoon but the channel has a trickle of people joining.
I'd like to encourage people to join and introduce themselves, drop thoughts etc.

2024-07-16 17:40:33

- reply

Ok I want to gauge how much interest there is around building a project like this.
I'm hosting an irc server at resistant.tech 6697 (self signed cert fingerprint 63:B1:2E:A7:8A:BD:8A:33:B5:62:21:7C:42:71:75:66:43:BA:D2:78:21:09:8E:80:17:7E:28:D2:58:65:E6:48)
There is a channel called #browser
If this is something you think should exist in some form them feel free to drop by and share thoughts.

2024-07-16 16:39:19

- reply

I think probably the best place to start would be to build a rendering engine that can support basic rss/atom feeds and a bare bones micropub client.
i.e. something minimally useful that covers both creation and consumption.
Soon after that you would probably want to start thinking about an extendable architecture to support activity pub variants, custom creation flows as well as taking small bites at rendering linked documents.

2024-07-16 15:53:44

- reply

I often think about the visions put forth by browsers like Amaya and, much later Flock.
That a browser should be a tool for creation as much as consumption.

2024-07-16 15:48:40

Over the last decade-plus web standards have become over-saturated to the priority of commercial interests.
To maintain a web browser to any kind of quality assurance and security you need a well funded team - or rely on one in your dependency tree.
Money rarely comes without strings attached and, on every page load, you can feel that tension between the pure philosophical vision and expected returns.
Perhaps the time has come to reassess what a web browser should be.

2024-07-16 02:19:05

- reply

It shouldn't need to be said but...you don't get to skip getting informed consent just because it would be difficult or time consuming or annoying or expensive.

2024-07-16 02:15:45

- reply

I think this quote by the Mozilla CTO demonstrates the issue quite well:
"I honestly don't think the uproar would have been avoided by a modal, and we would have been interrupting the lives of hundreds of millions of people with a choice that is at best time-consuming to evaluate and at worst (and most commonly) entirely inscrutable." - https://old.reddit.com/r/firefox/comments/1e43w7v/a_word_about_private_attribution_in_firefox/lddphvk/?context=3

2024-07-15 23:59:25

- reply

I think reasonable people can disagree on the nuances of privacy engineering here...you can get really in the weeds of epsilons and trust boundaries.
Mozilla need to understand that that isn't what is happening here.
The fact is this is a new data gathering vector, it doesn't matter how privacy-preserving it is, it should be subject to *new, informed, and proactive* consent - rather than being automatically enrolled in an experiment.

2024-07-15 20:51:14

- reply

Firefox is really important. I want it to continue to be a viable independent browser, and a viable base for alternative browsers.
I still think it is the best option, but it continues to get harder and harder to make that case. And I really want Mozilla to deeply reflect on why I, and many others, feel that is the case.

2024-07-15 20:29:21

- reply

Maybe in an alternate timeline, where the mozilla good-faith budget had not been severely diminished by years of questionable policy and technical decisions - there might be a legitimate argument to made here, with adequate signposting, for maybe putting forth some version of this experiment to investigate a possible harm reduction measure to surveillance advertising.
We don't live in that timeline.

2024-07-15 20:25:57

Thoughts on the new Mozilla statement regarding #firefox #PPA
I don't care how strong you think the privacy properties of a new feature are (and there are legitimate arguments to be had disputing those technical claims) - enabling an experiment like that by default is incredibly disrespectful to users, and doubling down is the act that comes across as outright hostile.

2024-07-15 18:54:55

- reply

Any reasonable deployment of any system is going to rely on a combination of all three kinds of privacy.
But the mixture of those types of privacy, and the assumptions that underlie them make a HUGE difference in the actual secrecy of the data being exchanged and processed - and the probability of privacy being violated.

2024-07-15 18:47:21

- reply

(btw those systems determine the outcome of real elections in the real world - this is probably fine because there are other mitigations in place , but I really wish people cared more about that kind of stuff)

2024-07-15 18:44:45

- reply

Reminds me of all the e-voting systems that use "mixnets" but operate the mixnet on the same physical computer operated by a single entity - completely misunderstanding the point of a mixnet.

2024-07-15 18:43:13

I really love that "privacy-preserving" as a descriptor has become so generic as to be meaningless.
A few real interpretations I've seen this last week:
- "we don't keep the data we log - promise"
- "we use oblivious http - don't ask about who operates the indirection node - ok it's us but it's legally separate..promise"
- "yes, we log and process individual records, but only to check correctness - then we throw that data away - promise"
To be clear: none of those are privacy preserving.

2024-07-09 06:03:01

- reply

Constraints also solve the main problem I was facing with prototyping such some tooling which was "how do you model a requirement"
To which I think the answer is "through a combination of natural language and linked constraints"
And that is actually tractable!

2024-07-09 05:51:58

- reply

Constraints are useful for automating some ongoing requirements analysis and they also help relieve some of that tension between specification and implementation.
And I think it it important to consider them in that support capacity - they are not the specification, but they do help shape it.

2024-07-09 05:46:00

- reply

A few people have pointed out that it feels like tests/interfaces etc. should have a place here...and I think they are correct.
Addendum: Constraints
Requirements also motivate zero or more constraints.
A constraint is defined as any limitation on the implementation of the system (e.g. a test is a constraint, so is an interface, or a type - they all limit the ultimate expression of the implementation)

2024-07-09 04:58:49

- reply

I'll probably spend some time prototyping out some tooling within this model, if anyone has ideas/suggestions particularly around (lightweight) requirements modelling I'd be interested in any references/ideas in that space.

2024-07-09 04:28:52

- reply

The distinction between a requirement and a task is a fuzzy one. There isn't a clean line - and I don't think there can necessarily ever be one.
But I also think that determining where that line is - is a critical part of systems engineering. It will evolve with your understanding of the problem - and right now I think our existing tooling (inspired by agile methodologies) ignores this aspect entirely.

2024-07-09 04:18:37

- reply

And there will always be this tension between specification and implementation.
Any sufficiently complete specification inevitably takes on the flaws inherent with implementation.
But we can still make rough cuts, deciding if a bug is challenging design/deployment assumptions or revealing new ones.

2024-07-09 04:12:11

- reply

I really don't like the bundle-of-tagged-issues model that is prevalent across the space - even when tightly moderated, and corralled into projects and milestones there is never a cohesive view of -what the software should do-
And when it comes to all of the things I really care about when building or reviewing software "what should the software be doing" is really the core question.

2024-07-09 04:08:51

A model of software development I've been playing with:
- all software starts with a set of requirements
- requirements motivate sets of tasks
- when complete, tasks progress one or more requirements.
A forge should primarily be a way to receive, moderate and apply a standard set of modification suggestions:
- feature requests that originate or modify requirements
- bug reports that jeopardize one or more requirements
- pull requests that resolve outstanding tasks

2024-07-02 17:56:50

I owe much of my philosophy on open source project management to the writings of the late Pieter Hintjens and, apropos of other discussions, I think this blog post in particular worth revisiting.
http://hintjens.com/blog:106

2024-06-28 22:10:00

- reply

Or to put it more plainly.
There is so much bullshit now, that it is impossible to even talk about how bullshit something is, because doing so requires disproving an ever growing pile of supporting bullshit.

2024-06-28 21:48:47

I'm not sure how to deal with topics where the debated models of reality are so fundamentally irreconcilable that even if I thought others were acting in good faith, I see no way of bridging the gulf between them.

2024-06-26 19:29:48

- reply

I know where the hesitation comes from, the last few years has seen no end to examples of previously open spaces being enclosed and the contributions there mined for the profit of a few.
After watching the flowers you plant be ripped up, packaged, and sold for the nth time, I think it's natural to reconsider your whole approach to floriculture.

2024-06-26 19:17:39

Putting words to feelings:
I spent the last decade+ putting things out into the public domain: code, research, security bugs etc. and overtime I feel I benefited from those actions 10-fold - both from the relationships I developed, and the tools and research and I had access too from others who did the same.
Recently, I've found myself more hesitant to release new things. My projects folder is full of prototypes/ drafts/notes that at one time I'd have put somewhere public.
And that sucks.

2024-06-22 18:42:23

Some thoughts on funding specific features/bugfixes in FOSS projects.
The fastest way is to do it yourself and submit a PR. This is true even if your PR is rejected.
The most cost effective approach is unrestricted funding to the project itself and trust experts to effectively prioritize.
For restricted funding: The cost to fund any particular feature = cost to do the work now + cost not to do other work now.
(The second term is usually much much higher than most people think reasonable)

2024-06-20 17:37:58

"it's a protocol" in the historical sense: there is an agreed upon written that details packet structure, packet flows and expected participant behaviour. There are a myriad of independent implementations.
"it's a protocol" in the modern sense: a very active repository containing a "reference implementation" that serves as a de-facto common dependency because whatever it does is the correct behaviour, and it's far too complicated for anyone to consider maintaining an independent implementation.

2024-06-17 21:20:03

- reply

We've been a little quiet on the release front in recent months because most of the team are busy ensuring they can survive in the economy - Open Privacy does not have the budget we once did.
But we are still making forward progress, there will be a new release out towards the end of June with some exciting new experiments, and we are all still determined to make Cwtch the best we can.

2024-06-17 21:16:00

- reply

Personally, I like it that way. I never want to be in the position where I, or anyone on the Cwtch team, can be coerced to make Cwtch less secure.
This tech isn't perfect, but it exists and it works. With time and effort (and funding) we can make it even better.

2024-06-14 01:28:25

- reply

This is your last chance.
After this, there is no turning back. You take the blue pill - the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill - you stay in Wonderland and I show you how deep the rabbit hole goes.

2024-06-14 01:24:38

write programs that do one thing well
(but to make things easier feel free to require build-time dependencies on small parts of multiple large monolithic code bases, don't forget to throw in at least 2 meta-build systems that require interpreting configure scripts in 2 different runtimes)

2024-06-10 19:39:18

- reply

The trend towards people not reading long documents and instead relying on automated summaries makes me uneasy.
So much of what is written is already sanded down from the raw data.
Decisions have already been made about what to include and what to leave unsaid.
I think it important to consider the words you use and how you structure them.
Every sentence is important, if it isn't then I would have deleted it prior to sending.

2024-06-10 19:31:43

At some point we dreamed of computers helping us build a better world
now your handwriting can do your algebra homework
and the computer can keep rewrite your emails to sound less aggressive
while also rewriting emails from others, because you didn't need to read all of those words
and now, you can get on with the important things
like asking the computer to do art for you
#apple

2024-06-04 19:09:08

- reply

At the end of the day, I'm the kind of person that compiles (and occasionally writes) my own kernels - this affects me to the extent that people and organizations I engage with use these awful machines - and I expect they will in droves.
I've long given up on the idea that any systems besides my own can be trusted to keep secrets - but I will keep trying to both build better ones, and encourage others to do the same.

2024-06-04 18:58:55

- reply

The boundaries could have been cut dozens of different ways, but they are where they are because of the compromises built into our systems.
And every paper cut compromise has led us to a place where modern Windows prevents you from taking a screenshot of Mickey Mouse while it happily subverts every other kind of process and workflow isolation.
That was and is a choice.

2024-06-04 18:46:51

- reply

The whole thing is a damn policy choice that's been playing out over 20+ years.

2024-06-04 18:43:28

- reply

For the record I totally understand that everyone from chip manufactures to browser vendors made the decision to sell out their own customers and users to support and implement DRM everywhere - I recall those days pretty well.

2024-06-04 18:37:59

After writing this note on Recall (https://mastodon.social/@sarahjamielewis/112482021770758791) a few weeks back, I've received many messages under the assumption that I don't understand how DRM / OS interaction works.
As if the integration of a broken, backwards technology into the core of our computing systems happened by accident.
"No, you see the OS doesn't get to see those bits of the screen, so it totally makes sense why the system scraps your financial documents and passwords but not netflix" - utterly unhinged worldview

2024-06-03 19:08:18

- reply

@npub1lw0…y9r7
llama.cpp has support for grammar based sampling, see: https://github.com/ggerganov/llama.cpp/tree/master/grammars

2024-06-03 17:44:13

For all the discussion of "prompt engineering" and "finetuning", I think the most interesting biasing structure for modern AI that has flown somewhat under the mainstream discussion is the ability to directly constrain the output space through e.g. grammars for llms and control nets for image generation.
It's weird to see people deploy the raw output of large scale generative statistical models when there are pretty powerful tools just sitting there that allow more finegrained application.

2024-06-03 05:58:30

- reply

It would be nice if politicians spent less time dreaming up imaginary ways to increase internet surveillance and more time thinking about how to reduce wealth inequality or carbon emissions.

2024-06-03 05:39:51

- reply

Though on the subject of client side scanning, the best approach I've ever seen was the Apple one; an impressive result of years of research.
It was fundamentally broken in any sane risk model that these tools are being proposed for.
https://pseudorandom.resistant.tech/neuralhash-collisions.html

2024-06-03 05:17:06

- reply

I don't know...am I supposed to seriously consider a call to mass subvert encryption? backdoor all open source code? pretend math doesn't exist? insist we never teach the children about mixnets? let this knowledge only be discussed in hushed tones?
I refuse to believe these never-ending discussions and statements and draft bills are nothing but unserious people doing unserious things.

2024-06-03 05:11:08

The thing about chat control / upload filters / client side scanning, whatever it's being called now; They are responses to an old generation of technology - one of an internet governed by centralized corporations.
Anonymous, peer to peer, file sharing exists. No centralized place to subvert - except the software running locally. Imperfect now, but intrinsically extant.
What proponents of these laws really want is to roll back the clock; something that is, fundamentally, not possible.

2024-05-29 23:39:18

- reply

There are plenty of things left to explore (I've documented a few at the end of the note) - and in order to make it useful for something like web search a few other parts are needed (and I am not yet convinced that other approaches aren't better suited to the problem).
That is to say: expect some more things extending off of this as I get around to working on them.

2024-05-29 23:22:05

New Paper: On the application of Bloom Filter Hierarchies representing
Sub-word Token Bigram Occurrence to Probabilistic Full Text Search
This is a note regarding a prototype I've been working on for a few months in the domain of Decentralized Search (and Indexing)
It covers a data structure with interesting properties that I've been playing with, and documents some experiments regarding naive full text search performance.
Comments/questions/critique welcome.
PDF: https://sarahjamielewis.com/decentralization/search/ftsbloom.pdf

2024-05-27 22:17:13

- reply

Example: I have these really cool (I think) search prototype but to be somewhat practically useful I need to write something I've termed a "query optimizer" for it (forget what you think those words mean in the context of search)
I've got some notes on how that might look, but I really need to give myself permission to sit down for a week and focus on that.
I have at least 3 other projects in similar states.

2024-05-27 22:11:46

*Looking at the pile of paper and code drafts filling up my hard drive and the pile of notes consuming my physical journal*
Maybe June will be the month I actually polish and release some things...

2024-05-27 20:31:15

- reply

There are certain signals that can be identified as minimizing the likelihood that something is spam:
- having minimal formatting / plain text representation
- minimal references outside of the core semantic domain of the document (e.g. no links to ad servers / no affiliate links)
- maximal referencing of other documents that are unspam-like
Nothing completely flawless, but I'm reminded of xkcd 810: https://xkcd.com/810/

2024-05-27 20:23:58

- reply

The concept of a spam filter is one from a more innocent age where even if spam was a majority of documents, it could still be identified and dropped.
I'm not sure it's possible to really identify spam anymore. Even previously well-trusted news publishers are playing games with thinly veiled advertorials / scientific journals are full of generative spam etc.
That problem is just going to get worse.

2024-05-27 20:18:38

The more I think about search engines and compiling and weighting corpora, the more inclined I am to implement hard signal-filters i.e. assume all documents are spam to start with and only accept a document into the corpus if it can be shown to be unspam-like.

2024-05-26 22:09:46

- reply

To answer a few common questions I've received in replies / dms:
- I might open source this at some point, though it won't be until I've done an optimization pass on the UI rendering / used it for a little while myself.
- It's written in Go (with some C for framebuffer/clipboard/mouse/keyboad stuff ), and my plan is to only support desktop linux.

2024-05-26 21:48:53

- reply

Made a little more progress on my graph software this afternoon. Nodes can now be edited, the graph can be saved (each node is saved as an individual file)
I also added clipboard interaction and support for image nodes, so you can now paste in an image and manipulate it like any other node.
https://files.mastodon.social/media_attachments/files/112/509/585/367/257/601/original/91f347dfdb000c21.mp4

2024-05-25 01:46:46

While understanding that not everyone has the kind of freedom that permits control over the systems they use...if you do have such freedom I encourage you to take advantage of it.
The most powerful thing about free and open source software is the ability to take it apart, understand how a piece of it works, and adapt it for your own purposes.
Don't like how something works? Rip it out. Share the modified version with the world.
Your systems don't need to be subject to the whims of others.

2024-05-24 02:03:01

Perhaps I have simply outgrown some kind of naive idealism, and perhaps some of it is the tendency to view the past through a more generous filter.
But wow is it hard to -find stuff- now. Even stuff I *know* exists. Hell, even stuff I know *I* wrote and put out there.
Lost in an ocean of empty words.

2024-05-24 01:55:44

Getting to the root of it, I think the thing I miss the most about the old internet was the unstated assumption that the people on the other end of the wire were...people who shared similar interests and just wanted to connect.
I think of all the friends I made, the experiences I had that branched from IRC channels / forums / and even twitter in the later days.
Now the main question I find myself asking of anything that comes across my screen is "what is this trying to sell me?"

2024-05-22 23:05:18

I spent large portions of my early career rearranging binary sequences on a chalkboard, and writing assembler for obscure architectures.
There are parts of my brain hard wired to recognize and align protocol stacks from a visual representation of a signal dump.
It's cute that you think you have to explain how computers work to me.