Mary Lee was the eldest daughter of Samuel Lee (of London) and Mary Piper, of Concord, Mass.
She and her four siblings all grew up in Concord in those years the country was contested and founded.

She married Oliver Williams, a hatter and merchant, in Concord in mid-September of 1796.

They took up housekeeping in Concord, living in a home the front of which also served as a salesroom for merchandise.
They went on to have 14 children, 10 boys and 4 girls, 9 surviving.

They were:
Ephraim Smith Williams (1802/1890)
Gardner Davinport Williams (1804/1858)
Caroline Lee Williams Stevens (1806/1847)
Mary Anna Williams Hodges (1807/1889)
Alfred Leonzo Williams (1808/1886)
Benjamin Oliver Williams (1810/1887)
Alpheus Fuller Williams (1812/1884)
Harriot Locady Williams Rogers (1814/1884)
James Monroe Williams (1817/1903)

The family became pioneers of Michigan Territory when they removed to Detroit in 1815, and thence to Waterford Twp. in 1819.
It was here Mary endured the many privations of pioneer life. After her husband died in 1834 at the home on Silver Lake, she after a few years moved to the Hodges House hotel in Pontiac to live with her daughter Mary Anna Hodges.

After the many happenings over the years in Pontiac and the many advancements in the city's position as a central focus of interest in Michigan, Mary passed in 1860.

A paper, speaking of the death of Mrs. Williams, says:

"Died at Pontiac, April 1, 1860. She was born January 11, 1777, in Concord, Mass. The last year of Mrs. Williams' life was one of intense suffering, which she bore with christian fortitude, in the full hope of a blessed immortality. Mrs. Williams leaves a large circle of relatives and friends to mourn her loss. For forty years she had been a resident of the county. From her position. Mrs. Williams became acquainted with many of the first emigrants and settlers of the county. Her kindness and courtesy to the early settlers won their esteem and none made her acquaintance who did not become her friends, and the attachment formed in the early settlement of the county remained unbroken during her life. For several years Mrs. Williams has resided with her daughter, Mrs. Schuyler Hodges, from whose residence the remains were yesterday removed and deposited in the Oak Hill cemetery beside her husband. Mrs. Williams was the mother of fourteen children, ten sons and four daughters, seven of whom only survive their parents: E. S. Williams of Flint; A. L and B. O. Williams of Owosso; Mrs. Mary A. Hodges, of Pontiac, and Mrs. Harriet L. Rogers, A. F. and James M. Williams, of California. Mrs. Mary Williams had forty two grandchildren and sixteen great grandchildren."