So we face a dilemma: how can we maintain the privacy benefits of local models while still accessing real-time information?
[…]
The problem is that this database is hosted on a server somewhere in the cloud. The host of the server can see our query and the response of that query, compromising our privacy.
A potential solution would be to download all the data locally. That way, we could query the database locally keeping the query and response private. But this is impractical – Twitter alone generates 200TB of text per year. Your phone can’t store that much data.
What we want is a way for the server to process our query without ever seeing what we were asking for or what results we got back. Let’s see if we can find a way to do that by looking at how search works.
How modern search engines work:
Modern search engines convert both documents and queries into normalized vectors such that similar items end up close together. This conversion is called “embedding”. For instance:
"I love cats" → A = [0.7, 0.71] (normalized)
"I like felines" → B = [0.5, 0.866] (normalized)
"Buy cheap stocks" → C = [-0.8, 0.6] (normalized)
Figure 4: Visualization of the three normalized vectors in 2D space. Vectors A and B point in similar directions (high similarity), while vector C is in a different direction to A (low similarity).
To find relevant documents, we look for vectors that point in similar directions – meaning they have a small angle between them. The cosine of this angle gives us a stable, efficient similarity measure ranging from -1 to 1:
cos(0°) = 1 (vectors point same direction)
cos(90°) = 0 (vectors are perpendicular)
cos(180°) = -1 (vectors point opposite directions)
The cosine similarity formula is shown below, where a and b are vectors with components a₁, a₂, etc. and b₁, b₂, etc., θ is the angle between them, · represents the dot product, and ||a|| denotes the length (magnitude) of vector a:
However, when vectors are normalized (meaning their length equals 1), the denominator becomes 1·1 = 1, leaving us with just the dot product:
cos(θ) = a·b = a₁b₁ + a₂b₂ + a₃b₃ + ...
This is why working with normalized vectors is so efficient – the similarity calculation reduces to simply multiplying corresponding elements and adding up the results.
Let’s compute the similarities between our example vectors:
The first high value (0.964) tells us vectors A and B point in similar directions – just as we’d expect for two phrases about cats! The second value (-0.254) shows that vectors A and C point in quite different directions, indicating unrelated topics.
This seemingly simple calculation – just multiplications and additions – is the key to private search.
Via X
Introducing EXO Private Search
Privacy-preserving web search for local LLMs.
Augments local LLMs with realtime search using Linearly Homomorphic Encryption: – Live data from X, crypto, Wikipedia (more coming) – 100,000x less data transfer than client sync – <2s round-trip time… pic.twitter.com/r6qXIiaD6z
Google DeepMind with Joel Z. Leibo, Alexander Sasha Vezhnevets, Manfred Diaz, John P. Agapiou, William A. Cunningham, Peter Sunehag, Julia Haas, Raphael Koster, Edgar A. Duéñez-Guzmán, William S. Isaac, Georgios Piliouras, Stanley M. Bileschi, Iyad Rahwan, Simon Osindero, and others:
What is appropriateness? Humans navigate a multi-scale mosaic of interlocking notions of what is appropriate for different situations. We act one way with our friends, another with our family, and yet another in the office. Likewise for AI, appropriate behavior for a comedy-writing assistant is not the same as appropriate behavior for a customer-service representative. What determines which actions are appropriate in which contexts? And what causes these standards to change over time? Since all judgments of AI appropriateness are ultimately made by humans, we need to understand how appropriateness guides human decision making in order to properly evaluate AI decision making and improve it. This paper presents a theory of appropriateness: how it functions in human society, how it may be implemented in the brain, and what it means for responsible deployment of generative AI technology.
I took it even further and rewritten it in C because in C there is no ready-to-use hashset and you need to make your own.
And with my 1 line hashing function hashset is still slower than (non-allocating unstable) quick sort because it doesn't allocate. pic.twitter.com/kR8AlxbcXc
