1.6 fixes

Great, I'm looking at it in detail tonight.

added 1 commit

6f01a449 - [fix] #1234 (closed) Remove ARM specific code

mentioned in merge request !1214 (closed)

I'll try it on my VPS today.

I have been running this branch on one of my member nodes for almost 24 hours and I am reproducing bug #1234 (closed):

2017-12-18T19:02:32+01:00 - error: WS2P >>> >>> WS ERROR: INCORRECT_PUBKEY_FOR_REMOTE
2017-12-18T19:02:32+01:00 - info: Matched 3 zeros 000BC8CE7E716E44D45D33C623F40D659A0EB3A9442B9D28F0FD3F126435F909 with Nonce = 20200000304596 for block#78675 by D9D2za
2017-12-18T19:02:43+01:00 - info: ✔ PEER BoZP6aqt
2017-12-18T19:02:45+01:00 - info: ✔ PEER GYEtpPnK
2017-12-18T19:02:47+01:00 - info: ✔ PEER 5fPevx21
2017-12-18T19:02:47+01:00 - info: ✔ PEER H7VemVkJ
2017-12-18T19:02:52+01:00 - info: ✔ PEER 2ny7YAdm
2017-12-18T19:02:56+01:00 - info: Matched 3 zeros 000666A09DB81CD6EC84C711481FD87AAC1974735662E60AC158FD7512044926 with Nonce = 20200000302533 for block#78679 by D9D2za
2017-12-18T19:02:58+01:00 - info: Matched 3 zeros 000F30FEA8B90E33F83BC980E312B7FFAF72CEC77AE544450DEA68A15C7AB6EF with Nonce = 20200000303011 for block#78679 by D9D2za
2017-12-18T19:02:58+01:00 - info: WS2PTOR: Could not connect to peer 2ZvEsd6s using `WS2PTOR txeyf3zs65sq2g3v.onion 80: WS2P connection timeout`
2017-12-18T19:02:59+01:00 - info: Matched 4 zeros 000034DEB5778AE81F590AE836BFF7907CB98D7B3942CCCD77484223BF4C3ADD with Nonce = 20200000303200 for block#78679 by D9D2za
2017-12-18T19:03:01+01:00 - warn: WS2P: cannot connect to incoming WebSocket connection: WS2P connection timeout
2017-12-18T19:03:14+01:00 - info: Matched 3 zeros 000F193494CBE51B2DDE1BD43FD8306724213D3AB0B707FA3EED3FB153288B6E with Nonce = 20100002439316 for block#78675 by D9D2za
2017-12-18T19:03:21+01:00 - info: Matched 3 zeros 0004B0F9940E81ED5D03644389ABE4CCD2229221C2C32EEDF3A34499A3EA80C7 with Nonce = 20100002393474 for block#78677 by D9D2za
2017-12-18T19:03:28+01:00 - info: WS2PTOR: Could not connect to peer 4tAz49Vt using `WS2PTOR x5mlxikgc6dazen4.onion 20901: WS2P connection timeout`
2017-12-18T19:03:29+01:00 - info: Matched 3 zeros 0008A8D24C616F9279AE8BBA63A57148724B9868B035C8C12C7E42C9E58DE008 with Nonce = 20100002395672 for block#78677 by D9D2za
2017-12-18T19:03:31+01:00 - error: WS2P >>> >>> WS ERROR: INCORRECT_PUBKEY_FOR_REMOTE
2017-12-18T19:03:32+01:00 - error: WS2P >>> >>> WS ERROR: INCORRECT_PUBKEY_FOR_REMOTE
2017-12-18T19:03:33+01:00 - info: Matched 3 zeros 000B6929BB58785E8662BCEE8DCB9F7F2A2C65FACD20D612AA6D3CF042953727 with Nonce = 20200000304451 for block#78682 by D9D2za
2017-12-18T19:03:38+01:00 - info: ✔ PEER 4364zZpz
2017-12-18T19:03:52+01:00 - info: Matched 3 zeros 000A0A0AAADA430656AF36F66AECB1605BDB13F9B8E14BF0998060B261727FB6 with Nonce = 20100002402248 for block#78677 by D9D2za
2017-12-18T19:03:52+01:00 - info: Matched 3 zeros 0004D3CB267B68C83367A32A347D3A682C0CEF0C63C9D553984A2EC1FE867079 with Nonce = 20100002402406 for block#78677 by D9D2za
2017-12-18T19:03:54+01:00 - info: ✔ PEER GfKERHnJ
2017-12-18T19:03:55+01:00 - info: ✔ PEER TENGx7Wt
2017-12-18T19:03:55+01:00 - info: Matched 3 zeros 000A5F9E5C31DFEEC763E1B5B36424FBAB27C47691455E4A46455DFCD9727E2F with Nonce = 20100002403299 for block#78677 by D9D2za
2017-12-18T19:03:56+01:00 - info: Matched 3 zeros 000C130DA47538C3A12DEBA169858968EF158DCE61ABD4215C46C6421E9421E4 with Nonce = 20200000301959 for block#78694 by D9D2za
2017-12-18T19:03:58+01:00 - info: WS2PTOR: Could not connect to peer J8aAWyZE using `WS2PTOR 3k2zovlpihbt3j3g.onion 20901: WS2P connection timeout`
2017-12-18T19:04:01+01:00 - info: Matched 3 zeros 0001DE9ED10C1DD302A574A3B82624D4963FAD99139F36E5BEE241765BD21CC3 with Nonce = 20200000303244 for block#78694 by D9D2za
2017-12-18T19:04:01+01:00 - info: ✔ PEER 7G2mS5aC
2017-12-18T19:04:03+01:00 - info: ✔ PEER VmGDNucM
2017-12-18T19:04:13+01:00 - info: WS2P: Could not connect to peer 5gJYnQp8 using `WS2P g1.aerisryzdvrx7teq.onion 53012: WS2P connection timeout`
2017-12-18T19:04:17+01:00 - info: ✔ PEER Com8rJuk
2017-12-18T19:04:22+01:00 - info: Matched 3 zeros 0008F8167C50C6FF0078B187B987890B6131465D33DE689C9A47E6CA96C912CB with Nonce = 20100002411157 for block#78677 by D9D2za
2017-12-18T19:04:23+01:00 - info: ✔ PEER 5dzkzedB
2017-12-18T19:04:24+01:00 - info: Matched 3 zeros 0002C232BA8BE2D7E226A28F92C7589170C3BF2928E5E90F31623C8799D35512 with Nonce = 20100002411559 for block#78677 by D9D2za
2017-12-18T19:04:46+01:00 - info: ✔ PEER 98wvdsHG
2017-12-18T19:04:46+01:00 - info: Matched 3 zeros 000A06F778C9E0D0B744E0DE8F9246B34C1BF0FA6643A7A70BC4DADC4D1DD4B5 with Nonce = 20100002358449 for block#78678 by D9D2za
2017-12-18T19:04:48+01:00 - info: ✔ PEER 9XEQzzSH
2017-12-18T19:04:49+01:00 - info: ✔ PEER 48SLtTLL
2017-12-18T19:04:52+01:00 - info: Matched 3 zeros 000F6477AA2064D603938001EDDD9A279D6138319FA0F23A6A2BF584F68C456A with Nonce = 20200000303053 for block#78699 by D9D2za
2017-12-18T19:04:53+01:00 - info: SIDE Block #78869-000002DE added to the blockchain in 11 ms
2017-12-18T19:04:53+01:00 - info: Block resolution: 1 potential blocks after current#78868...
2017-12-18T19:04:53+01:00 - info: Block #78869 added to the blockchain in 161 ms
2017-12-18T19:04:53+01:00 - info: Cancelling the work on PoW cluster of 2 slaves
2017-12-18T19:04:53+01:00 - info: Cancelling the work on PoW cluster of 2 slaves
2017-12-18T19:04:53+01:00 - info: GIVEN proof-of-work for block#78869 with 5 leading zeros followed by [0-1]! stop PoW for D9D2za
2017-12-18T19:04:53+01:00 - warn: The proof-of-work generation was canceled: Proof-of-work computation canceled because block received
2017-12-18T19:04:53+01:00 - info: Block resolution: 0 potential blocks after current#78869...
2017-12-18T19:04:53+01:00 - info: WS2PTOR: Could not connect to peer 5dzkzedB using `WS2PTOR hsfww4ttislofxob.onion 10901: WS2P connection timeout`
2017-12-18T19:04:53+01:00 - error: Unhandled rejection: WS2P connection timeout
2017-12-18T19:04:53+01:00 - error: WS2P connection timeout
2017-12-18T19:04:56+01:00 - info: Matched 3 zeros 000F0F8D5DFAADE18954A94A548404FD4F4A5584F47CCE190155918795606555 with Nonce = 20200000303695 for block#78699 by D9D2za
2017-12-18T19:04:59+01:00 - warn: Identity with hash 'BAE0773869DAD0CB9F334F2A80E9F96091BD4D15CE7CB13E4D414631F8C8385C' not found
2017-12-18T19:05:00+01:00 - warn: Identity with hash '51FA2547B28A63538E4C537387B4EB96CA2C3F80A05E92612985B5B6D052BAC6' not found
2017-12-18T19:05:03+01:00 - warn: Identity with hash '1DC360DBCEB2858B31E591BFFB821BB10DD9F2B102AAAAFDC258FFD550F5508C' not found
2017-12-18T19:05:03+01:00 - warn: Identity with hash 'D1B825D9374E16B8A6897DCECF0AE930D80B5EE25D4E2888724F90C08935F623' not found
2017-12-18T19:05:03+01:00 - warn: Identity with hash '1C8CCA7A0CB6E2DE80B77002137FED3303DF728A127B89C8A78EED67C43C9EE9' not found
2017-12-18T19:05:03+01:00 - warn: Identity with hash '83E8BEA5789B0C54A8F22F50B4AF5E94A88B05626CD3820AF69514A9964AD421' not found
2017-12-18T19:05:04+01:00 - warn: Identity with hash 'BF7251EA48EB3B5CE9EEEADC1E72FC2AE15D1CD229B2CE016AF0A0EA79329202' not found
2017-12-18T19:05:05+01:00 - info: Matched 3 zeros 00096E1D00F528F3D10B3E36C35FEAB6288B35E41F8A9211B56691F00368C10E with Nonce = 20100002362432 for block#78678 by D9D2za
2017-12-18T19:05:08+01:00 - info: Generating proof-of-work with 5 leading zeros followed by [0-1]... (CPU usage set to 100%) for block#78870 D9D2za
2017-12-18T19:05:08+01:00 - info: ✔ PEER FSAPnBZi
2017-12-18T19:05:14+01:00 - info: ✔ PEER HmH5beJq
2017-12-18T19:05:20+01:00 - info: Matched 3 zeros 000E7F5BB619BB988BD2BAF9FE96B87936D702F8452E2232C7C66FE63F0F013C with Nonce = 20200000302487 for block#78710 by D9D2za
2017-12-18T19:05:23+01:00 - info: Matched 3 zeros 000C00B4D9119577EFBC71CE969D51C67F1A515441F85D376FE876D4441B360D with Nonce = 20200000303007 for block#78710 by D9D2za
2017-12-18T19:05:24+01:00 - info: Matched 3 zeros 000488C1FC1BCA8068C3A63D8CAF69EE1A47B64B1CA3526EAFDF88EDCF6202EC with Nonce = 20100002366497 for block#78678 by D9D2za
2017-12-18T19:05:33+01:00 - info: Matched 3 zeros 0001E836CF2C3675B58DCBB3EF7D19C1E11C934FA58F0651B42BE488A420335F with Nonce = 20200000303102 for block#78711 by D9D2za
2017-12-18T19:05:34+01:00 - info: Matched 3 zeros 000BF71213EA515371B4C15D24400438ED31D3FFFA9408D12017380E1FB70300 with Nonce = 20200000303287 for block#78711 by D9D2za
2017-12-18T19:05:35+01:00 - info: Matched 3 zeros 0007A81E109C51B36ACCDBFE6CC5BA00AAC14BB4E5C68641711592748E4F2F7C with Nonce = 20100002369401 for block#78678 by D9D2za
2017-12-18T19:05:41+01:00 - info: ✔ PEER AUEbgiuW
2017-12-18T19:05:45+01:00 - info: POST Peer FEkbc4BfJu#78855 to 1 peers
2017-12-18T19:05:45+01:00 - info: ✔ PEER FEkbc4Bf
2017-12-18T19:05:47+01:00 - info: Matched 3 zeros 000A5B6FB6EFD1BBFE9B05CBE73908170D78A36DE521621ED2FD9F866E48878C with Nonce = 20200000303943 for block#78712 by D9D2za
2017-12-18T19:05:47+01:00 - info: Matched 3 zeros 000C830E617080FC65D4D79468E5F1DBCF4A927E90195832F6EB1E0693C31DDF with Nonce = 20100002373017 for block#78678 by D9D2za
2017-12-18T19:05:54+01:00 - info: Matched 4 zeros 000029479C4267252FA31EC91D8D24CF28566F12F2601174D888A28696EA194E with Nonce = 20100002374879 for block#78678 by D9D2za
2017-12-18T19:05:55+01:00 - warn: ETIMEDOUT
2017-12-18T19:06:04+01:00 - info: ✔ PEER Be1eVp7e
2017-12-18T19:06:07+01:00 - info: Matched 3 zeros 00091E07E10EA225157C41807DB933F4A8717D685D93170DCCAC650C86B315D9 with Nonce = 20100002378601 for block#78678 by D9D2za
2017-12-18T19:06:07+01:00 - info: ✔ PEER 5gJYnQp8
2017-12-18T19:06:10+01:00 - info: Matched 3 zeros 000F3968B1771EB187FBE7D346BFAEAF6087B9826390C3B93C2AB40BD3C04DA3 with Nonce = 20100002379337 for block#78678 by D9D2za
2017-12-18T19:06:11+01:00 - info: Matched 3 zeros 000A4840E4D9260B57B99C078B06E7AD7243E86F383A48D84448F6E5B8F13929 with Nonce = 20200000302148 for block#78723 by D9D2za
2017-12-18T19:06:16+01:00 - info: ✔ PEER LFoG1wup
2017-12-18T19:06:18+01:00 - info: Matched 3 zeros 000CA45EE099723AF70A504FF730C33F01A6C04005FC2E52273D4C41DC777FCF with Nonce = 20200000304066 for block#78723 by D9D2za
2017-12-18T19:06:25+01:00 - info: ✔ PEER 6FqjGokA
2017-12-18T19:06:31+01:00 - info: Matched 3 zeros 00085AB5D806AA1BE29C5B85C811515854D6816540C812FABFA885C76634ECE1 with Nonce = 20200000304878 for block#78724 by D9D2za
2017-12-18T19:06:34+01:00 - info: ✔ PEER 3wdDzBz1
2017-12-18T19:06:35+01:00 - info: ✔ PEER 2ZPiZZLK
2017-12-18T19:06:36+01:00 - info: ✔ PEER HrpJvimW
2017-12-18T19:06:38+01:00 - info: Matched 3 zeros 0007AA6D4F370DDD408609366B46EA0ABD91B5EED1F562D7DEBF6B06DBC2C328 with Nonce = 20200000303650 for block#78726 by D9D2za
2017-12-18T19:06:49+01:00 - info: Matched 3 zeros 00058880E07FCACA424BCC25F09D1E6351AD77E8F0D783BBD44435BDAD79B6BB with Nonce = 20100002450938 for block#78674 by D9D2za
2017-12-18T19:06:50+01:00 - info: ✔ PEER 2sZF6j2P
2017-12-18T19:06:52+01:00 - info: ✔ PEER 4GX5gUFw
2017-12-18T19:06:52+01:00 - info: Matched 3 zeros 0004739DF6E9FBCD2D41CAEF64A82E6E5EC3E401806663E2DD16C86EA7AC61BC with Nonce = 20200000304678 for block#78727 by D9D2za
2017-12-18T19:06:54+01:00 - info: Matched 3 zeros 000D4E4E2D99A2AA06E914455F6183A050EFD21D4DF2702D13CD0EA55A165925 with Nonce = 20100002452162 for block#78674 by D9D2za
2017-12-18T19:06:57+01:00 - info: Matched 4 zeros 0000DA73F06C1A3DE8054675E90C84BB4880579AF2BBCC284421258EA790D79F with Nonce = 20200000303224 for block#78728 by D9D2za
2017-12-18T19:06:58+01:00 - info: Matched 3 zeros 00090575B7CFCAD3F91A0866B706330A176EA9A08B8D30A61B9827E3C2E61815 with Nonce = 20200000303514 for block#78728 by D9D2za
2017-12-18T19:07:00+01:00 - info: ✔ PEER Do99s6wQ
2017-12-18T19:07:09+01:00 - info: ✔ PEER 7BLFqMo2
2017-12-18T19:07:10+01:00 - info: Matched 3 zeros 000122E49A8FC1DD925E002A77428D440C0357AF51F2164D25F935B78C034D5C with Nonce = 20200000303838 for block#78729 by D9D2za

2017-12-18T19:07:16+01:00 - info: Matched 3 zeros 000719D0F6F1A796627A97337C2CA4FDFC13B0FBD6399C11D107006266FEE292 with Nonce = 20100002458255 for block#78674 by D9D2za

I am also running this branch.

Everything works fine for me for now.

@librelois, the bug you are reporting exist since a while. I have already reported many months ago. For me, it's not a blocking bug for 1.6 release. It happen to me when doing to much development manipulations. I don't know how to reproduce it. For now, if it's only you which is able to reproduce it, we should go for a release without waiting more.

@moul something has changed that makes this problem become a blocker (it may be something else because I've never had anything similar in the past).

This bug occurs systematically after a few hours, and on several different machines !

mentioned in issue #1234 (closed)

After a restart the bug #1234 (closed) reoccurs immediately, and besides it is necessary to manually kill the Pow processes :

2017-12-19T00:39:04+01:00 - info: Matched 3 zeros 0008120558C3346E46D20ECF8135A23F4CDFC1EF391EAA447A2BBE1E5DCEA3F7 with Nonce = 20100000003441 for block#78894 by D9D2za
2017-12-19T00:39:07+01:00 - info: Matched 3 zeros 0006F3AAF01F42226FC0C5B28E0D1A1EA4714FB976431DDE641FCA430BBBF10A with Nonce = 20200003093979 for block#78890 by D9D2za
2017-12-19T00:39:13+01:00 - info: ✔ PEER 5cnvo5bm
2017-12-19T00:39:24+01:00 - info: Matched 3 zeros 0005F00658B3B69FF703EAFA368BCEAFD3DC43B7F8C0AAD91DD97CEC7D54BA0D with Nonce = 20200003098543 for block#78890 by D9D2za
2017-12-19T00:39:26+01:00 - info: Matched 3 zeros 0007BA468D3174E88D8D5BEE12F696CAE826D6291CF2FAD6C3E7679F22E68859 with Nonce = 20200003099316 for block#78890 by D9D2za
2017-12-19T00:39:27+01:00 - info: Matched 3 zeros 000D6D5A1D115AF7A0A956FD54DABCDD97AD536933E86ECE53C36C700B340849 with Nonce = 20200003099534 for block#78890 by D9D2za
2017-12-19T00:39:41+01:00 - info: ✔ PEER 81t4Ucqm
2017-12-19T00:39:48+01:00 - info: Matched 3 zeros 000D151DCC7351053A8D92FC03C54F61CFC5CEB7009A9FEBA4029CBCCF457E88 with Nonce = 20200003105214 for block#78890 by D9D2za
2017-12-19T00:39:52+01:00 - info: Matched 3 zeros 000ED196EDA1F7F7715737750B608B1565C8054BEDCAA69BF63585C4F315F8C5 with Nonce = 20100000004134 for block#78899 by D9D2za
2017-12-19T00:39:58+01:00 - info: Matched 3 zeros 0005EF43A3348133541C09489A3AA59D82D3361861338BF619E7B23DEDE600B6 with Nonce = 20100000004021 for block#78929 by D9D2za
2017-12-19T00:40:01+01:00 - info: Matched 3 zeros 000B127A93FB40A5F5B41742DB62FCD36B55B785EFFFAD201F8E907C9411F664 with Nonce = 20200003109485 for block#78890 by D9D2za
2017-12-19T00:40:04+01:00 - info: Matched 3 zeros 000304A33FBD0080EF1D894CEA76C48FD872B3B1E73D19151AFF5427236D701B with Nonce = 20200003110118 for block#78890 by D9D2za

I was also reproducing this bug killing manually PoW processes. I am using this branch the whole day long and I wasn't able to reproduce this bug.

Here is the ticket about same issue: #649 (closed)

@moul yes you're right by the way now I can't reproduce this bug it's really mysterious. Maybe it was linked to my dev environment. @c-geek What do you think? We deploy a pre-release to see if this bug occurs in other members ? (we can always revert to a new pre-release if necessary)

Yes, let's have more tests on this one with a pre-release. And if it becomes really too much blocking, we will revert this feature.

What is annoying is that this bug could be easily fixed using Shared Memory Buffers, but this is a Node10 feature. So not before 1 year!

Note: I've just thought about some refactoring that could help having a better control over the PoW workers. I will try this idea and make a new MR soon (which means tonight at best).