machine learning - LSTM vs. Hidden Layer Training in Tensorflow -

i messing around lstms , have conceptual question. created matrix of bogus data on following rules:

for each 1-d list in matrix:

• if previous element less 10, next element previous 1 plus 1.
• else, element sin(previous element)

this way, sequence pretty based on previous information. set lstm learn recurrence , ran train on lists 1 @ time. have lstm layer followed connected feed-forward layer. learns +1 step easily, has trouble sin step. seemingly pick random number between -1 , 1 when making next element when previous 1 greater 10. my question this: training modifying variables in connected feed forward layer? why can't learn non-linear sin function?

here's code snippet in question:

lstm = rnn_cell.lstmcell(lstmsize) y_ = tf.placeholder(tf.float32, [none, os])  outputs, state = rnn.rnn(lstm, x, dtype=tf.float32) outputs = tf.transpose(outputs, [1, 0, 2]) last = tf.gather(outputs, int(outputs.get_shape()[0]) - 1)  weights = tf.variable(tf.truncated_normal([lstmsize, os])) bias = tf.variable(tf.constant(0.1, shape=[os]))  y = tf.nn.elu(tf.matmul(last, weights) + bias) error = tf.reduce_mean(tf.square(tf.sub(y_, y))) train_step = tf.train.adamoptimizer(learning_rate=1e-3).minimize(error) 

the error , shape organization seems correct, @ least in sense learn +1 step without crashing. shouldn't lstm able handle non-linear sin function? seems trivially easy, guess set wrong , lstm isn't learning anything.