“Sex using force” in Timor-Leste – the perils of survey questions

My last post on domestic violence and marital rape attracted lots of interesting comments. Some points made are valid, others are red herrings, and I have already addressed many in the comments section. I don’t have time to cover everything here, but I will address the most important point, which is the reliability of the data with regard to language and understanding.

First however, I must clearly state that I make no apology for ‘sensationalism’. My blog post was primarily about violence against women, which is a serious problem in Timor-Leste and needs to be talked about more. Yes, there is some good news in there too, and I will write further posts about that, e.g. Timor-Leste appears to be on-track for the water supply MDG. But that blog was about violence against women: so, to the survey questions.

A survey designer has a lot of control over how questions are supposed to be asked by the enumerator (i.e. person carrying out the survey), and the response options that are given. However, the designer can never be completely sure how the enumerator (i) asks the questions in practice, or (ii) interprets the response. Furthermore, the enumerator cannot be sure whether the respondent has (i) understood the question, (ii) given a truthful response. These are the perils of surveys, but we have to live with them – they are the only source of important demographic data.

For this reason, a lot of effort goes in to training the enumerators (i.e. the people who do the fieldwork) to ensure they fully understand the questions. It is clear from p.6-7 of the DHS 2009/10 (click here to download) that they took pains to:

   1. translate and back-translate the questions into Tetum and Bahasa
   2. pretest the questions in 3 different districts
   3. hold a four-week training course in Tetum for the enumerators
   4. conduct oral and written tests of enumerators

We can assume that the questions were asked in whichever of Tetum or Bahasa the respondent was more comfortable with, but this is still problematic (see language post). According to the 2004 census, 43% of Timorese speak Bahasa, and 91% Tetum of some kind (but only 46% Tetum Prasa, the Dili version). We cannot therefore be sure that all respondents understood all the questions perfectly. Did the enumerators pass over households that spoke neither, use a local translator, or do something else? However, we can be sure that with language being such a problematic issue in Timor-Leste, the training will have tackled these problems head-on and proposed ways around them. This is why I accept the data as presented.

[Update 28/2/11 – I worked out a way around one of the language issues. We can safely assume that men with at least primary education can understand either Bahasa or Tetun, at least to the extent required to understand the below question. Using the table published in the report, it is easy mathematically to isolate only those men with greater than primary or greater than secondary education. Ignoring the differences across districts for a moment, we find that that on average across the country, those with greater than primary 1 in 7 agreed, and with greater than secondary 1 in 8 agreed. The rest of this post has not changed.]

Now, to the controversial question, which you can read on p.389 of the survey (link here again). “Do you think that if a woman refuses to have sex with her husband when he wants her to, he has the right to use force and have sex with her even if she doesn't want to?” The question is unequivocal about (i) him wanting to, (ii) her not wanting to, (ii) the use of force.

Want and force are simple concepts, not e.g. complicated names of diseases or health issues, which are often a source of confusion in DHS surveys. The options for answers are “yes”, or “no” or “don’t know / depends”. In the survey report we are only given data on “yes”, so we cannot infer the split between “no” and “don’t know / depends”.

It is notable that the answers to this particular question show significant variation between districts. More than 30% of men in 3 districts (Baucau, Bobonaro and Covalima) said “yes”, but in 6 other districts less than 5% said yes, with the other 4 districts falling somewhere in between. This might lead us to suspect that the question was not universally understood in all districts.

It might be worth investigating these kinds of problems regarding other survey questions. The only other such example that I have looked hard at is the data on diarrhoea on p.139. Reported rates of under-5s having diarrhoea in the last two weeks are unusually low in 3 districts. Furthermore, some findings are counterintuitive, with mother’s education apparently contributing to higher diarrhoea incidence.

In conclusion, I should state that I stand by what I wrote, which was merely commenting on published government data. I was not saying that all Timorese men are rapists. Violence against women is a big problem both in Timor-Leste and in other countries around the world. It needs higher priority.

http://www.timor-leste-violence.org/pdfs/Timor-Leste-Violence-IB5-ENGLISH.pdf

http://www.amnesty.org/en/campaigns/stop-violence-against-women

http://www.etan.org/issues/women.htm

1 in 6 Timorese men think it’s OK to rape your wife if she refuses sex

Yes you read that right.

[update 28/2/11 - this post was since followed up with a supplementary post exploring the survey questions involved: to read it, click here. The text of this post has not changed]

This statistic comes p.219 of the Timor-Leste Demographic and Health Survey 2009-10 (DHS) which came out in December. It was carried out by the National Statistics Directorate of the Ministry of Finance, with funding from various donors, you can get it from www.measuredhs.com, click here and download the PDF. There's a useful 3-page summary at the beginning which gives you the headlines.

Many governments now carry out a DHS every 5 years or so. As WaterAid’s resident data nerd, I’ve had the joy of reading about 20 different DHS from various countries over the last few years. Like any survey, it has its flaws. Judging by some of the data on sanitation and diarrhoea there have been some serious translation issues (sympathy due here – Timor-Leste has 30+ local languages, check out my language post). However, of all the types of surveys I’ve worked on, DHS is the strongest: large nationally-representative sample, well-tested questions, and a consistent methodology.

So, 1 in 6 Timorese men think it’s OK to rape your wife if she refuses sex. That’s just one of the depressing statistics in the section on gender equality and domestic violence. Over a third of women have experienced physical violence since the age of 15, and this rises to over half of women in Dili.

64% percent of women (yes, women) believe that a husband is justified in beating his wife if she goes out without telling him. The figures for men are similar and, in fact, women seem to be more willing to sanction domestic violence than men. There’s no space to go into detail on this here – look at p.213-14. Even more depressingly, these views don’t appear to change much in younger women.

Clearly, a lot more work needs to be done on both stopping violence against women, and changing social attitudes so that women themselves see such violence as unjustified. Of course this will take time, but we can be sure that changing social attitudes will take more than posters of Horta and “hapara violensia kontra feto”. Not having much knowledge in this area, I will let the data do the talking and leave the solutions up to the experts (if you work in this area, please leave a comment).

These were the most shocking statistics I read in there, but it’s not all bad news. It’s also worth noting that most of the data is more hard facts than attitudes. For example, under-5 mortality has gone down a lot. It declined from 83 per 1,000 live births during the in 1999-2003 to 64 per 1,000 live births during 2005-2009, which is a decent chunk out of MDG 4.

Less welcome is the data on undernutrition (anecdotally, one reason why publication was delayed for so long…) – 58% of under children under 5 are stunted and 45% are underweight. That’s high compared to similar countries, and is one reason why Timor urgently needs to sort out its food security problems. Of course, nutrition is about far more than food, and I’ll take the opportunity to bang the drum for sanitation here, which is strongly linked to undernutrition, both through its impacts on diarrhoea and on acute respiratory infections – for more see p.6 of this report

I’ve obviously been poring over the DHS data on water and sanitation in a lot of detail and comparing it to previous surveys – a full post on that to follow later. So, I’ll end with a link to a bizarre article on DHS from Tempo Semanal. Often I like TS articles, but this one was a desperate attempt to add political intrigue, headlined with a year–old unrelated quote from Horta complaining about donors always spending money on reports. They evidently failed to notice that (i) this survey was led by the government, (ii) without these surveys the government wouldn’t be able to target resources and policy responses.

In conclusion, I urge you to read the 3-page summary at the beginning of the new DHS (again link here). It really contains a wealth of statistically solid and nationally-representative information about development in Timor-Leste.