Skip to content

Conversation

Kenzzer
Copy link
Member

@Kenzzer Kenzzer commented Mar 2, 2025

Here's how numArgs is retrieved

std::vector<DataTypeSized_t> &argTypes = callingConvention->m_vecArgTypes;
size_t numArgs = argTypes.size();

Here's the loop

size_t firstArg = 0;
if (callConv == CallConv_THISCALL)
	firstArg = 1;
for (size_t i = 0; i < numArgs; i++) {
[...]
void *regAddr = callingConvention->GetArgumentPtr(i + firstArg, m_pDetour->m_pRegisters);
memcpy(paramAddr, regAddr, size);

Consider x86GccThiscall and function void Foo::Test(void)

x86GccThiscall::x86GccThiscall(std::vector<DataTypeSized_t> &vecArgTypes, DataTypeSized_t returnType, int iAlignment) :
	x86GccCdecl(vecArgTypes, returnType, iAlignment)
{
	// Always add the |this| pointer.
	DataTypeSized_t type;
	type.type = DATA_TYPE_POINTER;
	type.size = GetDataTypeSize(type, iAlignment);
	type.custom_register = None;
	m_vecArgTypes.insert(m_vecArgTypes.begin(), type);
}

So

std::vector<DataTypeSized_t> &argTypes = callingConvention->m_vecArgTypes;
size_t numArgs = argTypes.size();

Will return numArgs = 1

Let's go to the loop

size_t firstArg = 0;
if (callConv == CallConv_THISCALL) // true
	firstArg = 1;
for (size_t i = 0; i < 1; i++) {
[...]
void *regAddr = callingConvention->GetArgumentPtr(i + firstArg, m_pDetour->m_pRegisters);
memcpy(paramAddr, regAddr, size);

So callingConvention->GetArgumentPtr(0 + 1)

Here's definition of GetArgumentPtr (x86GccThiscall inherits from x86GccCdecl, and x86GccCdecl is an alias of x86MsCdecl)

void* x86MsCdecl::GetArgumentPtr(unsigned int iIndex, CRegisters* pRegisters)
{
	if (iIndex >= m_vecArgTypes.size())
		return NULL;
	[...]
}

iIndex is 1, m_vecArgTypes.size() is 1. nullptr is returned, this is then used on memcpy, this is wrong, it must be fixed.

@Kenzzer Kenzzer requested a review from peace-maker March 2, 2025 13:00
@Kenzzer Kenzzer closed this Aug 20, 2025
@Kenzzer Kenzzer deleted the k/dhooks_callconv branch August 20, 2025 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants