"a conceptually simple and computationally cheap extrapolation scheme strictly improves the worst-case convergence rate. […] improves the best possible worst-case performance by the same amount as conducting O(sqrt(N/log(N)) more gradient steps."
https://optimization-online.org/2024/02/on-averaging-and-extrapolation-for-gradient-descent/
Paul Khuong /
npub1an…lvzj5
2024-03-01 22:57:50
Author Public Key
npub1anc03agyz8usjs9l5eyeaxhkgu0wxuyhuvd3h9d7nwm9w2e2rraqtlvzj5Published at
2024-03-01 22:57:50Event JSON
{
"id": "0eba0112011ee22e877f54361b36546d8b37c1ca268c3dca009a748617de529b",
"pubkey": "ecf0f8f50411f90940bfa6499e9af6471ee37097e31b1b95be9bb6572b2a18fa",
"created_at": 1709333870,
"kind": 1,
"tags": [
[
"proxy",
"https://discuss.systems/users/pkhuong/statuses/112022904553895886",
"activitypub"
]
],
"content": "\"a conceptually simple and computationally cheap extrapolation scheme strictly improves the worst-case convergence rate. […] improves the best possible worst-case performance by the same amount as conducting O(sqrt(N/log(N)) more gradient steps.\"\n\nhttps://optimization-online.org/2024/02/on-averaging-and-extrapolation-for-gradient-descent/",
"sig": "301f44517416710559d87613e7914bef1efa9f4331dc7fc680e7599cd69cb61a6cc5899af5b2f8a5670a780e2ed1f97ef6aa9fda34fff5e2197ab1648c7983c7"
}