As I mentioned, it is hard to consider Abraham an ungodly man in the ethical or spiritual sense.  There would need to be some evidence that Paul was using “ungodly” as a description of any fallen human being whether an unbeliever or not.  But Paul appeals to David in close proximity:

And to the one who does not work but believes in him who justifies the ungodly, his faith is counted as righteousness, just as David also speaks of the blessing of the one to whom God counts righteousness apart from works:

“Blessed are those whose lawless deeds are forgiven,
and whose sins are covered;
blessed is the man against whom the Lord will not count his sin.”

Is this blessing then only for the circumcised, or also for the uncircumcised?

But Psalm 32 refers to the godly in a way that would go against Paul’s point if he were using the term in this way:

Blessed is the one whose transgression is forgiven,
whose sin is covered.
Blessed is the man against whom the Lord counts no iniquity,
and in whose spirit there is no deceit.

For when I kept silent, my bones wasted away
through my groaning all day long.
For day and night your hand was heavy upon me;
my strength was dried up as by the heat of summer. Selah

I acknowledged my sin to you,
and I did not cover my iniquity;
I said, “I will confess my transgressions to the Lord,”
and you forgave the iniquity of my sin. Selah

Therefore let everyone who is godly
offer prayer to you at a time when you may be found;
surely in the rush of great waters,
they shall not reach him.

But what if Abraham means by “ungodly” the Gentiles?  This would make sense of where Paul is going (v. 10: “How then was it counted to him? Was it before or after he had been circumcised? It was not after, but before he was circumcised.”).  And it would avoid a conflict with Psalm 32 as well as his own description of Abraham’s behavior.

It would also answer what I think is John Murray’s strained attempt to distance Abraham from Phinehas:

• Genesis 15.6: “And he believed the Lord, and he counted it to him as righteousness.”
• Psalm 106.30-31: “Then Phinehas stood up and intervened, and the plague was stayed. And that was counted to him as righteousness from generation to generation forever.”

If Paul is making the reference he will emphasize about Paul being a Gentile when “counted righteous” then the distance is immediately clear.  Phinehas was a circumcized believer who, in his zeal for the covenant, slew a Gentile and a compromised Jew for unlawful intercourse.  Phinhas was a Jewish covenantal hero.  And Paul is pointing out that Abraham had the same status as Phinehas, and presumably greater status as the called forefather of Israel, simply by believing, even though an uncircumcised Gentile.

(If one contemplates this comparison, I think it will become clear that God responded to Phinehas action precisely because it demonstrated faith in the presence of corporate unbelief.)

I think this way of thinking makes Psalm 106 work with Paul’s argument rather than being a complication that has to be dealt with.

But what of the move from justification of Gentile Abraham to the justification of David.  Paul has already provided an understanding for this move in Romans 2:

For circumcision indeed is of value if you obey the law, but if you break the law, your circumcision becomes uncircumcision.

So justifying uncircumcised Abraham and sinful David are both related.  Paul has already also quoted Psalm 51.4 in Romans 3:

Against you, you only, have I sinned
and done what is evil in your sight,
so that you may be justified in your words
and blameless in your judgment.

So David has by killing Uriah and seducing Bathsheba put himself outside the people of God (note the uncleanness concerns in the psalm) but God can bring him back in.  And Paul’s whole argument has been that Israel is corporately apostate and thus no different than the nations.  Rather, Israel with the whole world is weak and ungodly (in the full sense of that word), and it was precisely at that moment that Christ died for us (“For while we were still weak, at the right time Christ died for the ungodly.”).

Later, however, he is going to point out (Romans 11) that God can graft Israel back in again.  Bringing up David’s sin and repentance is a good way of making a case before explicitly making the argument.