freamon

Hi. Just looking at what lemmy.ca sends my server:

From nginx log:

your.ip.address - - [20/Feb/2024:03:48:47 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.ca/"               
your.ip.address - - [20/Feb/2024:03:48:49 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca/"         
your.ip.address - - [20/Feb/2024:03:48:57 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca/"              

Dumping the JSONs reveals 3 duplicate files, all containing:

"id": "https://lemmy.ca/activities/create/0968bf93-ac05-4178-bf3f-28cdc65fc338"
"actor": "https://lemmy.ca/u/HairyOldCoot"
"object": {
  "id": "https://lemmy.ca/comment/7454502"            
  "content": "That was an episode of Sliders."
  "published": "2024-02-20T03:48:45.810068Z"

These are 3 Creates, not 1 Create and 2 Updates or anything. I've chosen a comment 'cos that's public info, but it's the same for all other activity too (Votes, Deletes, etc).

lemmy.ca isn't alone in this. endlesstalk.org sends everything twice (info posted here), as does lemmy.sdf.org and mander.xyz, but lemmy.ca is the only one I've seen send stuff 3 times. Other instances (e.g. aussie.zone, feddit.nl, lemm.ee, lemmings.world, lemmy.world, programming.dev, reddthat.com, sh.itjust.works, slrpnk.net) don't, but there's nothing obvious like lemmy version numbers to indicate the difference.

Whilst I'd prefer you didn't send me stuff 3 times, I'm mostly mentioning it because I doubt it's good for your own resources to send every activity by every lemmy.ca user out multiple times. Also, the OPs of my community are from Mastodon, so I imagine you're sending them the same message 3 times too ...

I just upvoted a comment in a community hosted on my site, and in the nginx log I can see:

ip.address - - [19/Feb/2024:14:33:31 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://endlesstalk.org/"
ip.address - - [19/Feb/2024:14:33:43 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://endlesstalk.org/"

Dumping out the activity, I can see:

... (single vote from account on another instance) ...                   
{
  "@context": [
    "https://join-lemmy.org/context.json",
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://endlesstalk.org/activities/like/f102d062-dfe9-4ece-96e8-310a09b249ba",
  "actor": "https://endlesstalk.org/u/freamon",
  "object": "https://sh.itjust.works/comment/9061222",
  "type": "Like",
  "audience": "https://lemmon.website/c/tails"
}
... (single vote from account on another instance) ...        
{
  "@context": [
    "https://join-lemmy.org/context.json",
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://endlesstalk.org/activities/like/f102d062-dfe9-4ece-96e8-310a09b249ba",
  "actor": "https://endlesstalk.org/u/freamon",
  "object": "https://sh.itjust.works/comment/9061222",
  "type": "Like",
  "audience": "https://lemmon.website/c/tails"
}
... (single vote from account on another instance) ...        

It's the same for comments too (if I grep the dump for other people's comments, there's only one, but there's two for comments made by me)

EDIT: can also see dupes from lemmy.ca (they actually sent the same Like 3 times).

cross-posted from: https://mastodon.social/users/MrLovenstein/statuses/111862356629002380

Secret Panel HERE 🃏 https://tapas.io/episode/2357963

You do not play for fun. You do it to inflict pain.

Usage for testing:
./get_latest_jsons.sh
./show_human_readable_timestamps.sh
If - for example - that showed that the latest JSON was 1691258704964.json:
./tcbot.sh TEST 1691258704964 0

This will show Community subscriber growth between the latest update and what happens to be currently be in the TEST folder (1691042690884.txt, from 2023-08-03), and then the data from "1691258704964" would become the new base, for the next update to compare its data to.

The REAL mode is what updates !trendingcommunities@feddit.nl

Requires bash, curl, bc, and jq. Don't think anything is particularly version-dependent, though.

Comments are welcome. I'm not too fussed about efficiency, as the bulk of it only runs once a day, but any suggestions for improvement are welcome.