232214
Technical notifications and up-to-date requests for action for TON validators, developers and integrators. If you're a validator or developer of a popular TON service you should be subscribed.
ANNOUNCEMENT: DECENTRALISED SYSTEM OF PENALTIES FOR POORLY PERFORMING VALIDATORS
The current system of penalising poorly performing validators will be fully operational next Monday, September 9.
How do validators determine that another validator has performed poorly?
The TON is supplied with the lite-client utility. In lite-client there is a checkloadall command.
This command analyses how many blocks the validator should have processed, and how many it actually processed in a given period of time.
If the validator processed less than 90% of the expected number of blocks during a validation round, it is considered to be performing poorly and should be penalised.
Technical description of the process: https://github.com/ton-blockchain/TIPs/issues/13#issuecomment-786627474
When and by whom a complaint is filed?
After each validation round (~18 hours), the validator stakes of validators that participated in that round are on the Elector smart contract for another ~9 hours.
During this time, anyone can send a complaint against a validator who performed poorly in said round. This happens onchain on the Elector smart contract.
All validators do not need to send a complaint.
How a complaint is validated?
After each validation round, validators receive a list of complaints from the Elector smart contract and double-check them by calling checkloadall.
If a complaint is validated, they onchain vote in favour of that complaint.
These actions are built into mytonctrl and happen automatically.
If the complaint has 66% of the validators' votes (by their weight), a penalty is taken off from the validator's stake.
What is the size of the fine?
The amount of the fine is fixed and equals 101 TON, which is roughly equal to the validator's income per round.
Where does this fine go?
The fine is distributed among the validators minus network costs and a small reward (~8 TON) to the validator who sent the correct complaint.
When was this functionality made?
This functionality was made in mytonctrl back in February 2021 https://github.com/ton-blockchain/TIPs/issues/13.
The complaints and fines functionality in the Elector system smart contract was made initially at the time of network launch.
Why didn't the penalty functionality work well earlier?
The network fee for sending complaints was significant, which as the Toncoin exchange rate increased, made it uneconomical to send them.
This has now been fixed by optimising the complaint message.
In a week from 9 September 2024, an automatic complaint sender will start working on several nodes in the network.
Is the system decentralised?
Yes, anyone can send a complaint, the penalty is only applied by a quorum of validators on the network.
There is no way for anyone to single-handedly fine anyone.
How do I prepare for the start of the penalty system?
Since this functionality is already implemented in node, you don't need to do anything to make the system work.
Please make sure you're complying with the validator guidelines.
During the week, we will publish additional tools and best practices for monitoring and maintaining the effectiveness of your validator.
Will the penalty system get stricter in the future?
Yes, the audience and the number of transactions in TON is growing rapidly and it is vital that the quality of work is at its best.
The system will improve and fines will increase this year. All updates will be announced in advance.
It makes sense to set up hardware, monitoring and validator work properly. If you don't want to do this please consider using staking services https://ton.org/stake.
Dear TON Validators,
We are releasing an important TON Validator Information Form today. This form is crucial for improving our network's performance and stability, especially in light of anticipated increased loads.
- The form collects essential data about your validator setup.
- Your input is vital for network optimization.
- Please complete the form today.
- Your prompt response will help us collaborate effectively on network improvements.
Access the form here
Your cooperation is critical for the TON network's continued success. Thank you for your immediate attention to this matter.
URGENT - Mainnet validators
Please update your nodes and then restart.
In mytonctrl run:upgrade stable_testnet
If you don't use mytonctrl manually switch node to stable_testnet branch.
If you have multiple validators you can update them all at once. Please stay tuned for a few hours, in case of subsequent instructions.
Mainnet validators with index <100
So far not enough validators restarted their nodes with correct flags.
If you didn't restarted your nodes with new flags yet, please do it ASAP.
Mainnet validators
Please check efficiency of your validators. In case of low efficiency or frequent crashes (including OOM) immediately contact @mytonctrl_help_bot
In particular we ask the following validators to check their nodes
2,AE083C661DAD64F734CCBF5A4BEDF398BC4CF5A6BF454E376BB4B4437CBB4C9CЧитать полностью…
9,D4AB33E3C1F558143BF63ECF82B26B6A1AD635149AFCDA508DFCF53DDEF49EA1
10,20ED0665410992AEC5F476CAC9D452D89B1C34C210C48C1D578D1B46A82A4088
65,481532E012CB8F7E1C1B179F52E31D9B4F20EFE1BB032196E2690975E5989729
88,F82905F3161A1F7108B4A3807FC5C970B2B5B45E58219C033F1614582DDEAAC5
90,95343C09F5D4F1830C8AE4C8577C66EE779FA723D0C8D10ACAFFAC9F346B1ECA
96,F1A0A4153857E5385884E4EFCE50FABBC00662F54139A58CAD6DCE97529F35F5
Mainnet Validators
Please be prepared to vote on Wednesday August 21 at 8:00 UTC for new transaction executor behavior, dispatch queue activation and increasing minimal split.
Details can be found here. Proposed changes will allow network to more evenly distribute load, improve stability of block generation and serialization process.
All validators MUST be updated to the latest version before voting. Target versions:
mytonctrl 7e90e26
validator 140320b
Mainnet Validators and Liteserver owners
Please update your node software (see "Target versions"):
update master
upgrade
7e90e26 (same for all kind of nodes)140320b
Validation slots
406 candidates participated in the last validator elections.
According to the network rules each round candidates are sorted by effective stake size and the first 400 validators are elected to become validators.
If you don't have enough stake to pass as sole validator, that is less than 355'000 TON at the moment, - please participate in network maintenance via staking https://ton.org/stake.
Mainnet validators
Proposal to increase gas limit for special transactions, in particular election, was accepted! Thank you.
Reminder Mainnet Validators
Please be prepared for vote tomorrow on Tuesday July 2 at 8:00 UTC.
Proposal will contain increase special_gas_limit from 35'000'000 to 70'000'000 in Config Parameter 20. In particular, this will allow Elector to conduct elections when more than 400 applications are submitted. Please note that according to the current configuration, the maximum number of validators is still limited to 400.
More details here.
Mainnet Validators and Liteserver owners
Please update your node software (see "Target versions"):
update
upgrade
--state-ttl 604800, so if you need more history (usually only liteservers need it) explicitly set state-ttl in systemd validator.service file.40daf3c / lite-server 55c3c0d 5c392e0
Scheduled network update on June 10
We are asking validators to schedule a time on June 10 at 9:00 UTC for validator software update.
This update is mandatory and, among other things, contains DB optimization, public overlay spam prevention mechanisms and optimization of block delivery.
Mainnet Validators and Liteserver owners
Tomorrow we expect drastic increase in network activity due to listing and minting of the Notcoin project, which is possibly the largest launch in the entire crypto industry in terms of userbase.
Please check you validators and liteserver software is up to date:
Target versions for node is 4cfe1d1.
Also, please check that your nodes and validators have enough free space on disk. If you have less than 100 GB, check section 5 here, as well as database grooming docs.
We ask you to keep an eye on the validators tomorrow and be ready to take action.
Mainnet Validators
Please be prepared to vote for block compression and transaction costs update on April 18 at 9:00 UTC.
Plea to validators
Tech details on proposals can be found here. Proposed changes will make block rate faster and transaction execution cheaper.
All validators MUST be updated to the latest version before voting. Target versions:
mytonctrl 9182448
validator 4cfe1d1
Important Instructions for TON Validators
Dear TON Validators, please follow these crucial steps to ensure network stability this week and to avoid new slashing penalties in the future.
Essential Actions:
1) Verify you're on the stable_testnet branch (commit 97c57c3). If not, update: /channel/tonstatus/130. For multiple validators, update one at a time.
2) Ensure your hardware meets or exceeds system requirements: /channel/tonstatus/102. Upgrade if necessary, one at a time.
3) We imperatively request you to use mytonctrl.
In your mytonctrl console:
- Update to the latest version: update master
- Enable telemetry: set sendTelemetry true
4) Set up monitoring dashboards for RAM, Disk, Network, and CPU usage. For technical assistance, contact @mytonctrl_help_bot.
DOGS Project Alert:
This week, the DOGS gaming project (50 million active users) is minting and listing, causing increased network load. We experienced two network outages on August 27 and 28. Please:
1) Stay available this week and respond within 1 hour, around the clock.
2) Closely monitor your hardware. Contact @mytonctrl_help_bot immediately if you need help.
3) Follow @tonstatus and be ready to apply urgent updates if necessary.
‼️ Updates on Slashing Mechanics for TON Validators:
The TON Core team is working on implementing of new slashing penalties for non-performing validators. Executing the actions above is crucial to mitigate the risk of losing rewards for validating the network.
Your cooperation is vital for maintaining network stability and TON's prosperity. Thank you for your prompt attention to these matters.
URGENT - Mainnet validators
Please make sure you have done the previous update. If you haven't please do.
Then update node flags:
1. open /etc/systemd/system/validator.service
2. remove -F 39987437:600844:7 -F 39987437:600845:7 if they are still present
3. add flags -F 39991868:601006:7 --catchain-max-block-delay 0.5 to the end of ExecStart
4. restart validator:
Читать полностью…
systemctl daemon-reload
systemctl restart validator
Mainnet validators
Block production is restored and stable now. Thank you for cooperation!
For now you can delete -F 39987437:600844:7 -F 39987437:600845:7 from ExecStart when it will be convenient for you. After that please run:
systemctl daemon-reload
Mainnet validators with index <100
Please restart your nodes with updated flags at 4:00 UTC
1. open /etc/systemd/system/validator.service
2. add flags -F 39987437:600844:7 -F 39987437:600845:7 --state-ttl 86400 to the end of ExecStart
3. restart validator:
systemctl daemon-reloadЧитать полностью…
systemctl restart validator
Mainnet validator
If your validator has index < 100 please be prepared to urgent action at 04:00 UTC (Wed Aug 28 2024 04:00:00 GMT+0000). Please set alarm.
P.S. Due to high recent activity (>20m transactions in recent 2 days), garbage collection overloaded many of validators for enough time for them to lost consensus with each other. To restore consensus back, validators need to be restarted at about the same time with specific flags.
Mainnet validators
Please take part in the voting for network config adjustment.
Details of updates are given here.
1. Check that your validator software is on the latest version: commit 140320b.
If you use mytonctrl:
1. Update mytonctrl via command: update
2. Vote for proposal:
vo 20308659988793623196998942342425699672407754574159059846525327984142483923860 21241086152080161891970114314094131332811466103307811578007752491905358303034
1.Create signed vote in validator-engine-console: createproposalvote 20308659988793623196998942342425699672407754574159059846525327984142483923860 vote-msg-body.bocЧитать полностью…
2. Send obtained vote-msg-body.boc to -1:5555555555555555555555555555555555555555555555555555555555555555 in internal message from any wallet from masterchain with 2 TON attached. If you are using wallet.fif script, it can be done via:fift -s wallet.fif <path-to-key> -1:5555555555555555555555555555555555555555555555555555555555555555 <seqno> 2 -B vote-msg-body.bocand send resulting message to network. If you are using lite-client, it can be done vialite-client -C global-config.json -rc "sendfile wallet-query.boc"3. repeat for 21241086152080161891970114314094131332811466103307811578007752491905358303034
Validators require 128Gb RAM
In a previous update, we implemented a Fast State Serializer, which reduces blockchain state serialization time from 18 hours to ~50 minutes.
This frees up more resources for the validator to work on validating transactions and blocks, which is necessary for network performance in general.
Fast serialization works only if there is enough RAM.
According to requirements validator should have at least 128Gb RAM. If you have less - please upgrade your hardware.
This message is for validators only.
Scheduled network update on August 12
We are asking validators to schedule a time on August 12 at 9:00 UTC for validator software update.
This update is mandatory and, among other things, contains introduction of dispatch queue and drastic improvement of state serialization process.
In the light of upcoming increase of network load we remind validators about importance of having strong machines, in particular having at least 128 GB of Memory.
Сritical vulnerability in OpenSSH
Recently a critical vulnerability in OpenSSH was discovered: https://ubuntu.com/security/CVE-2024-6387
Since most of validators work on Ubuntu/Debian releases and use OpenSSH, we recommend check and update software on validator nodes if necessary.
Mainnet validators
Please take part in the voting for network config adjustment.
Details of updates are given here.
1. Check that your validator software is on the latest version: commit 5c392e0.
If you use mytonctrl:
1. Update mytonctrl via command: update
2. Vote for proposal:
vo 22440293175314732041600029416833597878678297180899118901294602657772681323261
If you do not use mytonctrl, each round:Читать полностью…
1.Create signed vote in validator-engine-console:createproposalvote 22440293175314732041600029416833597878678297180899118901294602657772681323261 vote-msg-body.boc
2. Send obtainedvote-msg-body.bocto-1:5555555555555555555555555555555555555555555555555555555555555555in internal message from any wallet from masterchain with 2 TON attached. If you are usingwallet.fifscript, it can be done via:fift -s wallet.fif <path-to-key> -1:5555555555555555555555555555555555555555555555555555555555555555 <seqno> 2 -B vote-msg-body.boc
and send resulting message to network. If you are using lite-client, it can be done vialite-client -C global-config.json -rc "sendfile wallet-query.boc"
Mainnet Validators Urgent Action Required
Please be prepared to vote for new gas limits for special transactions July 2 at 8:00 UTC.
In recent weeks, the number of validators has grown to a level that threatens the normal conduct of elections. Limits must be updated to continue smooth operation. If you are planning to launch a validator soon, please postpone it until the end of next week.
Reminder
Mainnet Validators and Liteserver owners
Please be prepared for upgrade on Monday June 10 at 9:00 UTC.
Liteserver owners
1) Please update MyTonCtrl to the new major MyTonCtrl 2.0 version.
In MyTonCtrl please run:
update mytonctrl2
disable_mode validator
enable_mode liteserver
a4b8bf2.db505f4.upgrade master
db505f4.
Mainnet Validators
Please vote for for block compression and transaction costs update.
Plea for validators »
More info about voting »
1) Please check that your node software is updated to the latest version (validator should be on 4cfe1d1 commit). If your node is outdated, upgrade immediately via upgrade command in MyTonCtrl.
2) Please check that your mytonctrl is updated to the latest version (should be on 9182448 commit). If necessary, you can update your mytonctrl via update command.
3) To vote via mytonctrl you need to use the command:
vo 12569949138907918341069315485240244200950856934919263564437769934039235613720
vo 31401000267374798460859142836851320859735453480263346121138978843222681889801
vo 41296061520515347990487695386109651690420586907458454458768144735616166931033
vo 49537534789854976259071705210466523357997345897986090889040825189829923897628
vo 96491208725598318198215712468033934914881492234615821929017670238581734249187
Mainnet Validators and Liteserver owners
Please update your node software (see "Target versions")
Instructions for update - take into account liblz4-dev installation in section 1 and archive-ttl setting in section 5.
Target versions:
mytonctrl 9182448
validator 4cfe1d1
This update is mandatory for validators and lite servers.