When there is a need to fit a parametric model to a finite population using data from a complex sample, the exact likelihood, which incorporates both the survey design and the probability model for the finite population, can be extremely complicated, or even intractable. Design-adjusted, approximate likelihoods, that is, modified likelihoods which incorporate the sampling design, are often used as an approximation to the exact likelihood. The design-adjusted, approximate likelihood can be used for either frequentist inference, or after specifying a prior distribution, Bayesian inference through the posterior distribution. The goal of this paper is to compare the design-adjusted, approximate likelihood to the exact likelihood, and to study the accuracy of this approximation. In this paper, two examples involving binary response data, the first using cluster sampling and the second stratified sampling, are presented in which the exact likelihood can be explicitly calculated. It is shown in these examples that even under extremely informative designs, the design-adjusted approximate likelihood closely matches the exact likelihood, with accuracy diminishing only for very small sample sizes or proportions close to the boundary. When lack of available resources and time constraints encourage the use of a design-adjusted, approximate likelihood, it is recommended that extreme designs, like the ones illustrated here, be used to help ascertain the scope of the error.