Notes on future changes to node behaviour w.r.t. inserts -------------------------------------------------------- Address inserts properly in NGRouting. The sequence is: we send InsertRequests we get back an InsertRepluy then we send DataInsert and wait for an Accepted THEN we start sending the request chain is we send DataRequest, they send Accepted, they send DataReply toad_: might be safer not to report insert times, since they aren't really the same thing as requests and could skew results sanity: well... maybe but what if the node only has inserts? toad_: well... if for some reason it gets lots of them? toad_: ultimately we need a more enlightened way to estimate and report inserts they'll be skewed by two things... 1) asymmetrical transfer rates, and 2) the extra message involved where can i wget the latest build with NGR ? yeah but generally we need to route inserts to the node with the best request performance emu10k1: http://freenetproject.org/snapshots/freenet-exp-latest.jar rename to freenet.jar thank you sanity: well, sometimes we will have a lot of inserts and few requests (but only on transients) the attack is for the node to be good at requests but swallow inserts i have no idea how to fix that the time required for inserts should be proportional to tDnf short of making the insert a full cycle thing, inserting to one node then pulling it out another one at the FNP level that'd be cool ultimately we could have a completely separate set of RTEs and estimators to deal with inserts but a significant change yes, we could, but then what? inserts at the start would be crap and would remain so for a considerable period well, inserts at the start would be routed on the basis of tDnnf tDnf even well lets see we send InsertRequests we get back an InsertRepluy then we send DataInsert and wait for an Accepted THEN we start sending the request chain is we send DataRequest, they send Accepted, they send DataReply so ok, so tDnf can estimate the time between InsertRequests and InsertReply the first phase, from us sending an IReq to us getting an IReply, will be roughly tDNF yeah the time between InsertReply and DataInsert will probably be proportional to tDnf too sorry, i mean the time between sending DataInsert and getting Accepted yeah, hopefully and the time to send the data will be proportional to tTransferRate the transfer rate will be completely different from any current estimator, because most links are asymmetrical yes, we track a multiplier - so tInsertTransferRate = M * tRequestTransferRate you think it would be constant per node? well - how complicated do you want it to be anyway how would we track a multiplier? well i mean, i am just trying to find something good enough to prevent abuse we could keep a separate transfer rate multiplier if we are looking for perfection you can start your PhD thesis today :-) but our current schemes cannot guard against abuse on inserts anyway inserts are down to trust, sadly exactly unless we change FNP so that we don't accept the insert is executed until we can fetch the data from an unrelated node my point is that if it weren't for the possibility for abuse we wouldn't have to worry about any of this so whatever solution we come up with only needs to be good enough to prevent abuse it doesn't have to be perfect well, the common attack would be to quickly but not too quickly swallow an insert without caching or forwarding it basically we need to remove any incentive for an attack to impose artificial delays on inserts requests are self-regulating yes, but if we do this inserts can be self-regulating too it just isn't necessary to go overboard ah point, if inserts are ignored, then there would be an incentive to serve them slowly exactly no, inserts can't be self regulating with the current FNP well, we can punish a node when it does something abusive requests can be because we get something concrete out of it which can't be faked oh, well - yes, that is true inserts CAN be faked * toad_ has concerns about hacked leech nodes that do this becoming popular we *could* address that by automatically requesting something we just inserted from somewhere else and tracking the probability with which it can't be found indeed which is already done by insert utilities but it's pretty slow given the current state of the network yeah, that sounds more like a client thing than something the node should do - but in this case the node would have to do it there is also the question of how stubborn we want to be, and if we regard it as a failure of the insert if we can't find it the problem with that is if we retry we might run into the node's alter ego the cost of not finding it is the cost of inserting it again but at least we know it served it once actually actually, the cost of not finding it is the cost of inserting it, requesting it, and inserting it again i suppose with premix routing, the alter-ego node can't tell it was us the cost of not finding it is the cost of a full insert like with requests we can keep a global estimator, like with inserts well, it is also the cost of the request to discover that it hadn't been inserted i just kind feel that this shouldn't be top of the NGR TODO list right now ...given that this threat has existed in Freenet from day one well, it's something to look at for the medium term. it's something that MUST be solved prior to 1.0 :) absolutely and has not really been discussed while I've been on the list (years!) definitely put it in the TODO i don't recall it ever really being discussed, but we have only really started to nail these things down with NGR it probably wouldn't be an actual FNP change but it would affect low level node behaviour i see no reason that FNP would need to chagbe change eve n well, the point is it can't be done entirely client level agreed * toad_ would post to the ML, but there seems to be a dearth of intelligent life there whenever I have something important to discuss lately Also, inserts are slowed down by the incoming data!! Maybe we should not log a transfer for the inserts at all?