Next I want to explore the /api/v1/instance/peers endpoint.
Before I start with my own data collection, let's take a look at the data @Gargron published half a year ago (https://gist.github.com/Gargron/48e67b1b14723cd178c951fe7f373a38).
pawoo.net and mastodon.social were clearly the most connected instances.
On average each instance was connected to 175 other ones.
Fediverse as of 2018-08-24.
7 788 instances with 1 303 382 connections.
Let's look at the different clusters. They aren't clearly separated, but still visible.
On the left in orange is the English and European cluster. 43%.
I don't see any subclusters.
At the top in violet are several niche instances. I would say the only thing they have in common is the non-technical focus. 17%.
Finally in green a little cluster (4%) of instances with domains ending in m.to. Again mostly Japanese.
If you know more, please let me know.
Searchable (ctrl+f) SVG without lines to keep the filesize reasonable: https://lucahammer.at/vis/fediverse/2018-08-24-fediverse-min.svg
If you compare it with the screenshots, keep in mind that it is mirrored.
Beware of over interpretation. This is no scientific work. It's just one way to visualize instances. There are many others and probably better ones.
Qualifying the connections by amount of follow connections could draw a different image.
In the end it's not important for your everyday use. You can follow every account of every instance (as long as your instance doesn't cut ties with the other one).
Most important are the people you follow and what you toot.
I tried to clean up the script, but it's still messy. Feedback welcome: https://gist.github.com/lucahammer/eb68fc43a34877a304e7b81e5f729c4b
Data as GDF from yesterdays visualization (zipped, 10MB): https://lucahammer.at/vis/fediverse/2018-08-24-fediverse-GDF.zip
@Luca This is brilliant work. Thank you very much for doing this.
@Luca People, there's work to do! We're in the middle of the "niche instances" with a "non-technical focus"... 😟
@hiemstra It's easily possible that my quick interpretations are wrong. Niche is a rather broad term. And I would say it's a good thing.
@Luca I think it's pretty accurate (but we're supposed to be a technical university, so...)
Nice work, BTW!
@hiemstra Thanks.
Academia will probably always be niche.
it's cool how many instances there are that i've never even heard of in the violet area
@Luca Oh, Eldritch Café is actually quite visible on the violet cluster!
@Luca How did you differentiate between the violet cluster and the others? I can't believe all the instances in the orange cluster are tech focused for example.
@Luca Cool. Now can anyone tell us what they're each about? It's really hard to find an instance that matches individual interests!
@darwinwoodka I didn't discover topical clusters and I believe it doesn't matter that much which instance you choose. I believe trust into the admins is more important than the theme.
@Luca have you ever ran a plot of instance age (if you save that data) vs nb connections (maybe relative nb connections)? curious to see how they‘re related
@halcy I only have data from those two dates. I fear that would make the plot useless/misleading. Maybe someone else has more data. I think https://mnm.social/ collected those numbers.
@Luca too bad. And instances don‘t report age :(
@Luca what? mst3k is American!
@Elizafox English as in language.
@Luca Oh I misunderstood, heh. Wonder why we got pushed to the right a bit. Might be cause I sometimes speak French.
@scarlett The SVG is searchable (Ctrl+f). https://lucahammer.at/vis/fediverse/2018-08-24-fediverse-min.svg
@Luca wow, I can actually see my self-run single-user instance from here! I feel, quite literally, extremely seen. 😁
@Luca tag yourself
@Cdespinosa @Luca This looks amazing. Will there be a possibility at some point to drill down or find a particular instance?
@fuzzface @Cdespinosa I created a SVG that can be search through (ctrl+f): https://lucahammer.at/vis/fediverse/2018-08-24-fediverse-min.svg (Sorry for flipped axis)
@Luca 😮 This is a cool visualization. Liking this place more and more by the day.
@Luca do you have a hi-res version of this available by any chance?
@djsundog Only 5000x5000px at the moment: https://i.imgur.com/kexSunA.jpg.
Will look into rendering something bigger tomorrow.
@Luca thanks!
@Luca Could you consider adding image descriptions to these posts?
@maloki Thanks for reminding me.
The images show names of instances that are connected through lines. They are one big hairball with two slightly visible groups of instances.
@Luca
Did you us umap or tSNE? Or something else entirely?
@vanderZwan #Gephi with ForceAtlas2 for the positions and Modularity for cluster detection.
@Luca
I admit I'm not too familiar with Gephi, but your description sounds like a "regular" force directed graph. If that already produces this, I'm curious what tSNE or umap would produce!
I bet even just feeding a 7788x7788 matrix with binary values based on whether they are connected or not would work well, it's kind of crazy how much structure those two algorithms can reveal!
@vanderZwan Which tools / libraries would you recommend? Preferably Python.
I will clean up my code and share the collection script as well as the collected data. Maybe you can use it.
@Luca
Well, there is only one real umap implementation at the moment, by Leland McInnes, who the guy who developed the algorithm. Luckily its in Python :)
https://github.com/lmcinnes/umap
Here he gives a talk explaining how it works in accessible terms at SciPy 2018:
https://www.youtube.com/watch?v=nq6iPZVUxZU
@Luca
Wow looks interesting, can You toot me an URL, that I can see it later on a bigger screen. On my 5 inch Xperia can't see so much ;)
@Luca
I like this🤔
.. thank you so kindly for posting❤️
Is there a time lapse so we can see the ebb. flow, migration, segregation, absorption, etc.. over the past 6 months, year, etc..?
Asking for a friend😏
@IncenseBerner I only have the data of those two dates, so there wouldn't be more to see than those two images.
@Luca Thanks for responding.. it was worth a shot to ask. Much appreciated.
@Luca That looks awesome! Is there a chance to get the graph data?
@hyperbolic Yes. I will publish it tomorrow.
@Luca Thanks for doing this. Very cool!
@Luca May I ask what is the green cluster at the right of the picture?
On the right side in turquoise is the predominantly Japanese cluster. About 35% of the instances.