A reader of Normblog presents a (standard) constructive argument for the counting numbers and then the infinite cardinals:

I happen to be friends with a jellyfish, called Jelly von Neumann. I asked Jelly about what Professor Atiyah said and she replied as follows…
‘Even if one has never seen any fish, crabs or the like, one may proceed as follows. First consider the empty set, { }, the set which has no elements whatsoever. Call that 0. Next, having got 0, consider the set {0}, whose only element is 0. Call that 1. Next consider the set {0, 1}, whose elements are exactly 0 and 1. Call that 2. Next consider the set {0, 1, 2}. Call that 3.
‘And so on. This gives you the infinite sequence 0, 1, 2, 3,… (One can prove that this sequence is infinite, since the operation involved is injective and never maps anything to 0.) You may even consider the whole infinite set, {0, 1, 2, 3,…}. Call this set omega. And you can go further. For consider the set {omega}. Call this omega + 1. Then consider {omega, omega + 1}, and call this omega + 2. Keep going. You get to omega + omega, and then omega + omega + omega. And so on. Eventually omega squared. Then omega cubed. And so on. Then omega to the power omega. And then (omega to the power omega) to the power omega. And then keep going. Eventually, you get to epsilon-zero. It gets a bit complicated after that. The point is that you can do mathematics just by virtue of thinking. Of course, I am a rather special jellyfish in that regard.’

Let us look carefully at the first few lines.  Before we have defined or constructed a single number, we are expected to have available a notion of a set and a notion of an element of a set.

First consider the empty set, { }, the set which has no elements whatsoever.

This is very odd – we are people who apparently know some set theory, but we cannot yet count (since we have not yet constructed the counting numbers).   And not just any set, but a set with no elements.   So maybe we can count!  How else can we tell that there are no elements in the empty set?  Perhaps we can only count zero objects.   And, moreover, this set is called “the empty set”, so presumably we know that there is only one of them.  There’s some pretty advanced set theory right there, in that casual statement of uniqueness, I would say.  (The claim of uniqueness, however, is not required for Jelly’s construction.)
Putting aside the question whether it is possible in principle for anyone, even those us with access to counting numbers, to count zero objects (arguably, counting is by definition an activity which requires the presence of at least one object to occur), let us continue with Jelly’s argument:

Call that 0.

So we can label objects.

Next, having got 0,

Wait a goddam minute, buster!  We just labeled an object “{  }” with the label “0“.    That is something different from getting or having anything.   And surely, in order to label an object “{  }” with a label “0“, we must in some fundamental sense already had had the label  “0“.   If we did not already have it, how else could we use it to label an object?   Jelly is using some pretty sleazy slight-of-hand here to slip from assigning a label that looks like a counting number to having the counting number itself, ready and able to be used for counting.   If the label we had used was (say) the greek letter alpha, then Jelly’s argument would proceed in exactly the same way as before, but we would not end the argument having defined the counting numbers.
Ignoring these problems, let us proceed:

consider the set {0}, whose only element is 0.

So now “0″ is an object, available for use as the element of a set. And we not only know some set theory, we ALSO know how to construct sets!   Just how do we do this?  Do we pick the object (or the label?) called “0″ and put it inside some curly braces?  How do we know when to start and stop picking objects?  For some reason we picked just one object.  Do we know how to count already?  At the next step we construct a set with two objects:

Next consider the set {0, 1}, whose elements are exactly 0 and 1. Call that 2.

From what collection of objects (or labels?) did we select the one called “0″ , or (respectively) the ones called “0″ and “1″? We seem not only able to construct sets and to count objects, but we also know how to select particular objects (not just any old objects, but particular ones) from some undefined collection of objects. Quite some skills we have here, we people who don’t yet know how to count.  And is the object that is here called “0” a different object with the same label as the one called “0” just three sentences before?   If they are different, how many of these different objects with the same label do we have?  And how can we tell them apart?  And, if they are not different, we must be re-using the same object called “0”.  Can we do this?  When last handled by us (two sentences before), the object called “0” was sitting inside the set {0}.  Can we just up and take it out and plonk it down inside the set {0,1}?  There are lots of deep questions here, questions whose possibly-different answers motivate entire branches of pure mathematics (e.g., linear logic, which deals with formal logics where we have available only a fixed and finite number of each mathematical symbol), which our jellyfish-cum-mathematician is glossing over or ignoring.
After a few rounds of this, Jelly hits us with:

And so on. This gives you the infinite sequence 0, 1, 2, 3,…

Well, no, actually. We never get an infinite sequence, since we, in this universe, can only ever complete a finite number of such steps in our lifetimes.  This is true even if all humans who ever lived, who are living, and who ever will live were to add their tuppence-worth of steps to the argument.  It’s hard to have confidence in a jellyfish claiming to construct a collection of infinite cardinals who can’t seem to distinguish between a finite and an infinite sequence.  At best (modulo the flaws identified above) we could get a finite, ever-growing sequence of counting numbers, a sequence that can be proven to exceed any pre-determined numerical threshold (thinking of these labels as real numbers for the moment), provided we allow sufficient time for the steps to be undertaken in the order described.  A finite, ever-growing sequence is not ever an infinite sequence; at best, we might call it potentially-infinite.
I think Mr Jelly ought to forget the peano lessons and adopt a cat.   And Norm, a Zimbabwean by birth, could perhaps remember how difficult it is to count objects in chiShona, with its ostentatious plenitude of noun-classes (21 according to Dale), and associated multitudes of counting words; urban Shona children nowadays usually count in English, even when they know little other English.
Reference:
D. Dale [1968]:  Shona Companion. Mambo Press, Gweru, Zimbabwe.  Second edition, 1972.