Walk a little farther down the block, to 8 St. Marks Place, once the site of a brownstone that housed the office of Madame Van Buskirk; one of the city's most famous and notorious abortionists in the 1860s and 70s.
8.
Throughout most of the 19th century, common law and courts allowed abortions before "quickening," that is, the perception of fetal movement, which usually begins around the fourth month of pregnancy. Although New York passed several anti-abortion laws during that time - the first in 1828 - generally this common law notion was either part of the enactment or conceded by the courts. Prosecutions were rare and convictions even more so. The attitude of most people was that before quickening, the termination of pregnancy was of little consequence. The question of abortion was neither a prominent social issue nor a personal moral quandary.
Beginning in the 1840s, abortion, like many other aspects of American society, became increasingly commercialized. This trend was prompted by, and in turn encouraged, a huge increase in the rate of abortions. Termination of pregnancy, once a desperate move by poor, unmarried women to avoid public shame, crept into the married middle-class as a means of family planning. Doctors began to specialize in abortions, do business relatively openly, advertise their cures for "female irregularities" and "menstrual obstructions" in the daily papers, and sell abortifacients by mail.
But in the years following the Civil War, a campaign spearheaded by the newly organized American Medical Association and plugged in the New York Times and several other newspapers (which began to pull their abortion advertising) resulted in a change in public and legal attitudes. Contributing to this shift was the sensationalized Alice Augusta Bowlsby case of 1871. Her decomposing body was found in a trunk checked at the Hudson River Railway Depot at West 30th Street and Tenth Avenue and traced to abortionist Jacob Rosensweig, who did business as Dr. Ascher at 3 Amity Place (now West 3rd Street) and 687 Second Avenue. Tried for the death of his young patient, Rosensweig was released on a technicality, an incident that caused outrage in the city and increased calls for tightening the criminal code. The very next year New York for the first time made abortion a felony. A modified bill was passed in 1881, and this law remained in force, virtually unchanged, until 1970, when abortion was in great measure legalized.

Alice Augusta Bowlsby

Jacob Rosensweig a.k.a. Dr. Ascher
During the laissez-faire decades of the 19th-century, the most famous abortionist in New York, indeed the country, was Madame Restell. She was so well known that the practice of abortion was sometimes referred to as "Restellism". Restell, whose real name was an Ann Lohman (nee Trow), came from England in 1831.
Her practice, which she began sometime around 1835, flourished until the 1870s, despite several arrests (which did little but give her free publicity), public censure, and newspaper exposes. She started in a modest home at 146 Greenwich Street and ended up a millionaire with a mansion on the northeast corner of Fifth Avenue and 52nd Street (next door, ironically, to St. Patrick's Cathedral, which was completed the year of her death). At its apex of 1870s, her empire included branch offices, traveling salesmen, and a mail-order business for her "Female Monthly Pills". On April 1, 1878, facing charges spearheaded by reformer Anthony Comstock that promised to put her out of business, she committed suicide.
After Restell, the most famous doctor in the city for "female irregularities" was probably Madame Van Buskirk, who practiced here at 8 St. Marks Place. Edward Crapsey, in The Nether Side of New York, published in 1872, mentions "Madame Van Buskirk, whose real name is Gifford, noted as one of the boldest and worst of her tribe, and whose den in St. Marks Place has long been known as one of the most infamous places in the metropolis".