How can associative learning be done on scalars in RBM since boltzmann energy func never has a minimum where node values are between their extremes
Boltzmann energy func = sum<x,y>(node[x]*node[y]*weight[x][y]
+ node[x]*weight1[x]) //and optionally with an offset per node
Normally sigmoid(weightedSum)=1/(1+e^-weightedSum) is used either as the chance for weighted coin flips of the bit states of nodes or their scalar values. I can get it to learn scalars this way, but it takes far more cycles than learning bits and is easily unlearned or cant remember nearly as many scalars. You'd expect it to learn log times less scalars since they have log number of digits, but its surprisingly bad at learning scalars compared to bits.
I need some energy function for the lower rbm layers that can be minimized by values between the extremes of 0 and 1, still tuned by weights, and (unlike the absval in radial basis networks) is smooth and acts differently in opposite directions.
Is this the energy for scalar nodes? I just combined the node update rule with leastSquares:
sum<x>( ( node[x] - 1/(1+e^-sum<y>(node[y]*weight[x][y])) )^2 )
Posted Jun 17, 2017, 11:20 AM UTC · edited