The paper supports a dialectical interpretation of Wittgenstein's method focusing on the analysis of the conditions of experience presented in his Philosophical Remarks. By means of a close reading of some key passages dealing with solipsism I will try to lay bare their self-subverting character: the fact that they amount to miniature dialectical exercises offering specific directions to pass from particular pieces of disguised nonsense to corresponding pieces of patent nonsense. Yet, in order to follow those directions one needs to allow oneself to become simultaneously tempted by and suspicious of their all-too-evident "metaphysical tone" - a tone which, as we shall see, is particularly manifest in those claims purporting to state what can or cannot be the case, and, still more particularly, those purporting to state what can or cannot be done in language or thought, thus leading to the view that there are some (determinate) things which are ineffable or unthinkable. I conclude by suggesting that in writing those remarks Wittgenstein was still moved by an ethical project, which gets conspicuously displayed in these reiterations of his attempts to cure the readers (and himself) from some of the temptations expressed by solipsism.