Ursinus:

Q73: Why then does the Holy Ghost call Baptism the washing of regeneration and the washing away of sins?

A73: God speaks thus with great cause, namely, not only to teach us thereby that just as the filthiness of the body is taken away by water, so our sins are taken away by the blood and Spirit of Christ; but much more, that by this divine pledge and token He may assure us that we are as really washed from our sins spiritually as our bodies are washed with water.

Commentary:

There are three reasons which may be assigned why the Scriptures thus speak, interchanging the names of the signs, aid the things signified. The first is on account of the analogy which there is between the sign, and the thing signified. The thing signified is according to its own nature, such as the sign is according to its nature, the opposite of which is also true for as water which is the sign, washes away the filthiness of the body, so the blood and Spirit of Christ, which are the things signified, wash away the pollution of the soul: and as the minister applies the sign outwardly, so God by virtue of his Spirit applies inwardly the thing signified to all those who receive the sign with true faith. Secondly, the Holy Ghost thus speaks for the confirmation of our faith through the use of the signs: for the signs used in the sacraments testify the will of God to us on account of the promise annexed thereto: ‘He that believeth, and is baptized, shall be saved.” But why does the Holy Ghost thus speak for the confirmation of our faith? Because in the proper use of the sacraments the exhibition and reception of the signs, and things signified, are inseparably connected. And hence the Holy Ghost interchanges the terms, attributing what belongs to the thing signified to the sign, and what belongs to the sign to the thing, to teach us what he gives, and to assure us that he does really give it. The third reason, therefore, why such language is employed is because the exhibition of the things signified, is inseparably connected with the signs used in the sacraments. Commentary on the Heidelberg Catechism, p 365.

Q74: Are infants also to be baptized?
Q74: Yes, for since they, as well as their parents, belong to the covenant and people of God, and through the blood of Christ both redemption from sin and the Holy Ghost, who works faith, are promised to them no less than to their parents, they are also by Baptism, as a sign of the covenant, to be ingrafted into the Christian Church, and distinguished from the children of unbelievers, as was done in the Old Testament by circumcision, in place of which in the New Testament Baptism is appointed.

Commentary:

2. The first end of baptism instituted by God is, that he might thereby declare and testify to us, that he cleanses those who are baptized by his blood and Spirit from all their sins, and therefore engrafts them into the body of Christ and makes them partakers of all his benefits. 2. That baptism might be a solemn reception or initiation of every one into the visible church, and a mark by which the church might be known from all other religions. 3. That it might be a public and solemn profession of our Faith in Christ, and of our obligation to faith and obedience to him. 4. That it might be an admonition of our burial in afflictions, and of our rising out of them and deliverance from them.

3. Baptism has the power to declare or seal according to the command of God, and the promise which Christ has joined to it in its lawful use; for Christ baptizes us by the hand of his ministers, just as he speaks through them.

4. There is, therefore, in baptism a double water; the one external and visible, which is elementary; the other internal, invisible and heavenly, which is the blood and Spirit of Christ. There is, also, a double washing in baptism; the one external, visible, and signifying, viz: the sprinkling and pouring of water, which is perceptible by the members and senses of the body; the other is internal, invisible, and signified, viz: the remission of sins on account of the blood of Christ shed for us, and our regeneration by the Holy Spirit and engrafting into his body, which is spiritual, and perceived only by faith and the Spirit. Lastly, there is a double dispenser of baptism: the one an external dispenser of the external, which is the minister of the church, baptizing us by his hand with water; the other an internal dispenser of the internal, which is Christ himself, baptizing us with his blood and Spirit.

5. Yet the water is not changed into the Flood or Spirit of Christ, nor is the blood of Christ present in the water, or in the same place with the mater. Nor are the bodies of those who are baptized washed with this visibly; nor is the Holy Spirit, by his substance or virtue, more in this water than elsewhere; but he works in the hearts of those who are baptized in the lawful use of baptism, and sprinkles and washes them spiritually by the blood of Christ, whilst he uses this external symbol as a means, and as a visible word or promise to stir up and confirm the faith of those who are baptized.

6. When baptism is, therefore, said to be the laver or washing of regeneration, to save us, or to wash away sins, it is meant that the external baptism is a sign of the internal, that is, of regeneration, salvation and of spiritual absolution; and this internal baptism is said to be joined with that which is external, in the right and proper use of it.

7. Yet sin is so washed away in baptism, that we are delivered from exposure to divine wrath and from the condemnation of everlasting punishment, whilst the Holy Ghost commences in us the work of regeneration and conformity wit11 God. Remissions of sins, however, continue to the end of life.

8. All, and only those who are renewed or being renewed, receive baptism lawfully, being baptized for those ends for which Christ instituted this sacrament. Commentary on the Heidelberg Catechism, pp., 372-373.