I'm speaking here as a teacher trainer.
Adaptations could be made depending on their prior schooling and literacy experience.
Good. If I understand correctly, you're talking here about what I'd call educational background. This is an area that includes all prior educational context. Generally speaking, this context ought not to be a consideration for assessment, though you're right that these things do have a considerable bearing on how a learner performs in class. Educational background can have a cultural component, but much of it doesn't.
Students might have learned literacy from a non-alphabetic language. This would that they might have more difficulty reading and decoding letters, meaning that they would need more instruction in that.
Yes, this is very important to remember, and often constitutes a key part of differentiation when it comes to teaching methodology, strategies, and general practice. It's not clear how you're trying to link this to assessment, however.
There are also differences between student’s religion, sex, geography, and other cultural factors that could play into their differentiation.
In my my view, the two most important aspects of a learner's profile are age (not cultural) and first language (very much cultural). Sex has nothing to do with culture. The geography of the place the learner is from has no bearing on learning success. Religion, though it is cultural, is something that I believe should definitely not be accounted for. Any particular special educational needs are important to take into account, but have no relation to culture.
It is important to make content and assessments culturally relevant and important. For example, if you are teaching students from Switzerland, it would be appropriate to make lessons and assessments on mountains since students would have background knowledge on this topic.
Let's stick to the point here, which is assessment, not teaching per se. I don't think you've made a convincing case that culture needs to be a factor in differentiating for the purposes of assessment, though you have made some sensitive points concerning learner profiling.