Source
r/ethstaker•
atrizzle

How many attestations do you miss?

Despite my robust at-home solo-staking setup, I still miss attestations occasionally for reasons seemingly outside of my control. I assume this is normal, but I wonder at what frequency others are missing attestations, too.

About how many attestations do you miss per validator per week?

I found my number by searching my email for "validator_attestation_missed" (the subject line from beaconcha.in's emails) and counting how many I've received in the last week, then dividing by the number of validators I'm running.

edit: considering the number of "0" responses, maybe these missed attestations are in my control...

View Poll

Posted Apr 19, 2023, 12:50 AM UTC · edited

731
atrizzleOP•

Those of you answering “0”, how?!

8
biggt76•

+1 Yes, please share your wisdom...I too would like to know

9
maninthecryptosuitStaking Educator•

Good hardware (NUC11i5, 32GB RAM, 2TB 970 Evo Plus), solid fast fibre internet (down/upload of at least 10 mbps, I am on symmetrical 500 mbps), reliable mains power, UPS for those once in a year momentary power blips I don't even notice but can reboot a PC, finally Lighthouse + Geth + MEVBoost.

I missed attestations only while I briefly tried Besu (it was buggy back then, they seem to have fixed it now) and a few sync committee attestations (but that's absolutely normal for sync committees).

That said, the goal is not to miss attestations - it hardly matters to the network or profit-wise. I just don't want to miss any proposals that's all.

2
biggt76•

Currently waiting for fiber. They buried the cable in my neighborhood about 3-4 months ago... they just haven't ran the cable TO my neighborhood yet....

2
maninthecryptosuitStaking Educator•

Nice, hope you get it soon! I used to have symmetrical gigabit but it was overkill, so downgraded to symmetrical 500mbps. Plenty fast!

2
biggt76•

I can't wait to get rid of Comcrap aka Xfinisuck.... I can get symmetrical 1 gig service for less than I'm paying for 600/20. Mind you I only signed up for 120 but they kept bumping my plan along with my bill

1
SgtRayTeku+Nethermind•

Same hardware setup with a Nuc11i7 and I miss at least 2-3 per day running Teku+Nethermind, not a huge hassle per se but I really wish we had stats about stability and attestations for each CL EL since that seems to be the only difference…

2
maninthecryptosuitStaking Educator•

Have you checked your block import times? If it's subpar then you know where to start. Could be execution client if all else checks out.

1
SgtRayTeku+Nethermind•

Mh since the last teku update, I’ve seen a few late block imports in my teku logs, I thought it was to do with the new 4ms MEV tolerance that had been implemented recently. I had the missed attestations before the update though so I think this is just a visible log change…

2
OkDragonfruit1929•

I am on symmetrical 5000 MB up\down ATT fiber

I wonder if the reason I receive at least 4-9 missed attestations per week is due to the fact I use a hardware RAID controller?

I think I will be building a NUC soon with a 4TB NVMe and 64 GB RAM, then re-provision the HP Proliant server as a media file server instead.

1
Only_Ad_7973•

Got new glassfiber internet

2
[deleted]•

[deleted]

1
Only_Ad_7973•

I miss 1 in 10000

1
ghostdunks•

Prior to the merge, I was running geth+lighthouse, I was running on pretty much 100% attestations, with the only blips being a result of occasional internet connectivity downtime every few months or so(to be expected with residential internet really), think I had like 7 missed attestations over a period of 12 months.

Then just before the merge, I transitioned from geth to besu, partly because of client diversity reasons and also because I was sold the idea that Besu didn’t need to be pruned(and no subsequent downtime because didn’t need to be pruned). Then merge happened and Besu was the cause of quite a few missed attestations but that stabilised after a few weeks, but since then I’m still seeing a few missed attestations a week, just randomly. My 11th gen i5 NUC is also overpowered for a validator, with 64gb of RAM and a capable SSD

And now that I’ve found out that even though Besu doesn’t require explicit pruning, the size of its database still increases a fair bit more than expected which will require a full resync to bring back down, which means even more downtime than a manual prune will with Geth. So I’m considering my options again and might switch back to geth.

5
maninthecryptosuitStaking Educator•

Geth will soon launch online auto pruning but consider Nethermind for client diversity.

2
-johoeTeku+Besu•

I think the merge has increased the number of missed attestations for everyone. Since the validators now not only have to validate the beacon blocks but also the execution blocks this is somewhat expected.

1
ghostdunks•

will find out soon enough. I'm going to sync up another execution client on a spare NUC using geth instead and use that to serve my consensus client. If that ends up with much better attestation performance, then that will tell me that its a besu-specific issue rather than a broader execution layer issue.

0
[deleted]•

I’m running geth-lighthouse on a high spec NUC and I seeing 98-99% effectiveness with several missed attestations per day/week even with high peer counts. It’s the network for sure. Things will improve over time, but irritating as it is, it’s really not a big deal.

1
SpacesiderStaking Educator•

I was missing quite a few when I was using Nethermind. I switched to Besu and my performance is so much better.

But I still miss a hanful of attestations per day, anywhere between 1 - 10 I think.

I already know my issue is because my SSD doesn't have the best IOPS, I will need to upgrade it at some point.

4
SgtRayTeku+Nethermind•

What CL do you use with Besu? I tried Teku+Besu when they released and had big performance issues, kind of go a bit better switching to Teku+Nethermind but still having at least 2-3 missed attestations per day, sometimes more…

2
SpacesiderStaking Educator•

I've got two full nodes, one is Lighthouse-Nethermind and the other is Teku-Besu.

I did have Lighthouse-Nethermind as my primary pair coming into the merge but I noticed I was missing a lot of attestations, I think Nethermind is heavier on disk resources compared to other execution clients out there so quite a few blocks weren't being processed in time because of the SSD it is installed on.

A few months after merge I swapped the endpoints around so Teku-Besu is now the primary and since then my missed attestations have dropped way down. I know Besu got a bit of a bad reputation around merge time because of the empty block issue, but it has come a very long way and performs exceptionally well.

I'm still missing a few attestations here and there everyday, but I have ordered an nvme drive and will be swapping over to that when it arrives which will bring a massive performance increase.

Sometimes you will miss attestations due to a late block proposals from other validators, so there is always the situation where a missed attestation is simply out of your control.

Increasing your peers can help out a bit.

2
SgtRayTeku+Nethermind•

Alright, thanks for your experience. I might try increasing the peers, I left them on 100 base since I started running, any idea how far I can go on a decent Nuc rig?

2
SpacesiderStaking Educator•

100 peers for each client is fine.

Sometimes you will just miss attestations due to things that are out of your control.

If you have done what you can on your end (Fast nvme, enough ram, enough CPU, adequate peers, decent internet) then at least any missed attestations will be reduced to a minimum.

1
issantarealmommy•

Probably jinxing myself but since the day after Shanghai upgrade haven't missed one attestation. Before that I'd say I'd average missing 1-3 per week.

3
atrizzleOP•

Sorry but you will wake up to the dreaded notification tomorrow

3
issantarealmommy•

Ah yes, that notification that is 99.9% bad news but has a .1% chance of being very good news. I know it well.

2
moooootz•

Yeah, same here. I don't know if it was the Shapella upgrade or the client upgrade that came with it. Perfect score since Shapella. Running a Rocketpool node with Nimbus & Besu in a VM.

Anyway, jinxed myself and preparing myself for the email tomorrow.

2
AutoModeratorMOD•

r/ethstaker strives for high quality interactions, our motto is "welcoming first, knowledgeable second", so please welcome every question and comment in this spirit. Participants who openly disregard this ethos will find their comments removed. This is a safe space for ALL Ethereum stakers, regardless of how they stake. Do not answer DMs offering help. DMs ARE SCAMS

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1
BrundleflyUrinalCake•

I think you want to say per node per week

1
atrizzleOP•

One node can support many validators, so I want to normalize results to the number of missed attestations per validator.

2
neobunch•

Sometimes you'll attest right before an empty block and then you will be delayed and in some odd cases completely miss the attestation, so those are completely out of your control.

If your execution client logs the time it takes to import each block, take a look at those messages. A healthy node with sufficient hw resources will import blocks in milliseconds, sometimes in the hundreds but always below 1 second. If some blocks are taking over 10 seconds to be processed that's a good bet for causing your missed attestations

1
oliveio•

3 missed, and I've been in Ukraine all this time.

  1. During a test attempt to relocate to AWS days before war when I underestimated sync time.
  2. During the first hours of war when I disabled my home machine but did not setup AWS yet.
  3. During blackouts, when I relocated staking back to a local rig.
1