Pages

Showing posts with label Urdu. Show all posts
Showing posts with label Urdu. Show all posts

Tuesday, December 13, 2011

SMS Content Filtering in Pakistan - The Real Story and Way Forward!

Česky: Logo programu k posílání SMS zpráv Esms...
Image via Wikipedia
*Note: The research by the author has also been published first here:
"What the *&~#" -1500 obscene words to block from SMSs or just 12 - while the PTA has yet to make up its mind, social commentators marvel at the obscenity of the very idea By Alefia T. Hussain, (The Daily News Sunday Magazine, Pakistan online and print).

The recent news about Pakistan Telecommunications Authority PTA directive to filter SMS content and its sudden decision to withdraw SMS filtration orders drove the world media into a viral frenzy provoking questions such as why did PTA attempt this move in the first place, who ordered them to do it from the government, how did they come about deciding whether the technology to implement such a filter was even there or not and what do they intend to do now? Questions like these have been pouring into my inbox since the issue and as promised, I researched deep into this issue.

I am undertaking some important social, legal, political and technical observations in to account here based on both logic and fact.

There are a number of issues here with a diversity of views and certain technology constraints. SMS abuse, SMS based abuse and SMS Spam has been widely perceived as an increasing concern for both regulators and mobile/cellular/telecom service users globally. In the case of Pakistan Regulatory Authority’s recent directive on SMS based content filtering actually may go beyond these prevalent concerns.

This whole incident lacks basic research!
First of all, as far as my technical knowledge and experience is concerned, the issue has not been assessed in detail in comparison to any existing global example and lacks baseline research. Secondly, total and effective SMS based content filtering against a keyword or a list of keywords as of yet is really not completely possible technically and technologically unless PTA is encouraged to publish its findings and the technical specifications of this in cooperation with the local Mobile/Cellular Operators that how are they going to actually implement this directive.

What triggered it?
According to my initial understanding, this SMS Content filtering activity being carried out by PTA may not have been triggered by any political pressure, political motivations or directions by the Cabinet Division. In fact, the Cabinet Division may now and should be encouraged to explore the concerns raised after this news got out from domestic and global media channels regarding PTA’s sudden content filtering on SMS. Furthermore, the global social media volcano has also exploded with extensive analysis and monitoring of the situation backed by both criticism and ridicule after the news spread through the Internet like wildfire.

Becoming the 1st to do so!
From my global experience dealing with Digital Censorship and Content Filtering, PTA may become the first telecoms regulator in the world attempting to implement such an extensive Content Filtering activity on SMS especially in two different languages but in the same English language script. Not all such news in this case is good news and may have negative implications towards how such regulations and their enforcements are construed by both regulators and citizens.

Why did PTA do it in the first place? Complaints?
So far it is widely being perceived that PTA may have solely issued this SMS Content Filtering directive for implementation by Mobile Operators after receiving complaints from consumers of cellular services filed with the Customer Complaints section otherwise there would be no other official or moral grounds to carryout such an order.

I have observed with many SMS users in the past that they are fed up with the anonymous and abusive messages they receive usually targeted at female members of their families and friends. This is a moral question indeed but maybe for the society to deal with it, not a telecoms regulator which itself is bound to legislative and parliamentary policy.

Upon observing both the English and Urdu Language keywords to be filtered lists that have been widely circulated across the Internet by many local and global media sources, many of the words mentioned in the Urdu Keywords Filter List may actually be a cause for many of these consumer complaints. Yes they are highly abusive terms and morally degraded language possibly used with intentions not actually asserting their meanings but as dual meaning phrases but again these may be morally disturbing for many.

Wrong assessment with no expert grounds!
As there is no public consultation neither carried out by PTA nor any psychological assessments through any official medical sources, PTA may be challenged on its assessment of socially or morally abusive terminology and the tools or mechanisms it uses to weigh immorality or public order both legally and by civil society groups. There needs to be some form of evidence what construes socially or morally abusive in the first place.

Has it incited language abuse?
Most of the words in the English list are actually catalysts of inciting deep concern among a broad range of social actors across the country. This raises concern that who were the English language/lingo experts at PTA that devised these lists and that they may be requested to be brought forward for being questioned publicly or maybe a case should be initiated with the courts under various human rights concerns of preventing access to usual and normal daily English language communication words.

The Keyword Filter Lists in both English and Urdu may require a revisit as well as more research. From the lists that I have gone through, there are many spelling mistakes so it is also evident that when the list was put together, it was not proof read nor it was actually read in detail by the PTA higher-ups. This raises some public transparency and accountability issues around how PTA vets its own processes before issuing directives and enforcement. Public Sector institutions and especially regulators should be very careful in how they propose and display information to the citizens.

No Oversight or Monitoring? No Public Consultations?
In reality, the Cabinet Division should be encouraged to question PTA’s officials that had led the creation of such lists and both the English and Urdu Language/Lingo discrepancies. It may be worth noting here whether religious experts, minorities’ representatives, principal English and Urdu Language advisors were brought together into a committee before designing such a list that may trigger unrest across the masses of the country? Socially this would be properly contested by legislators and courts in more developed nations.

If we study any single sided process that does not involve multistakeholder and public dialogue or even consultation with the Cabinet Division before implementation, usually such a process may only be initiated with the mutual understanding between PTA’s Chairman, DG and Member Technical and the only grounds that so far may make this enforceable are user complaints filed with the User Complaints section of PTA. I would believe that a great deal of thinking and survey would go into before issuing and implementing such directives.

Where's the technology? What, no technology yet?
Technically speaking and for the information of the general public, the SMS Content Filtering will be implemented by filtering software at the Telecom Operators end that actually receive or process and transmit all the sms messages being sent out to their users. I would first like to reference what is an SMSC to understand this process.

Technical Jargon 
(source:wikipedia) When a user sends a text message (SMS message) to another user, the message gets stored in the SMSC (short message service center) which delivers it to the destination user when they are available. This is a store and forward option. A short message service center (SMSC) is a network element in the mobile telephone network which delivers SMS messages.

An SMS center (SMSC) is responsible for handling the SMS operations of a wireless network. When an SMS message is sent from a mobile phone, it will reach an SMS center first. The SMS center then forwards the SMS message towards the destination.    The main duty of an SMSC is to route SMS messages and regulate the process. If the recipient is unavailable (for example, when the mobile phone is switched off), the SMSC will store the SMS message. It will forward the SMS message when the recipient is available.

What is the Validity period of an SMS message? 
An SMS message is stored temporarily in the SMS center if the recipient mobile phone is offline. It is possible to specify a cutoff period after which the SMS message will be deleted from the SMS center. Once deleted, the SMS message will no longer be available for dispatch to the recipient mobile phone (even if it becomes online).

Accordingly, any SMS that is sent from any mobile user is first stored on the SMSC while its delivery is delayed for a few seconds. Then the Mobile Operator can implement a character string search to match keywords against the content of the message and block the delivery of matching content. It simply removes the words and may sometimes give the message “Some text missing” or may simply display some blank spaces for content blocked at the receivers end.

Logical and Technical Bypassing of such filtering!
What PTA may have failed to consider are the deeper technical issues associated with this content filtering activity? Currently SMSC does not have the capability to actually filter out non-alphabet characters. Wild cards such as ***** etc can be used to actually make the scanning software application skip virtually all the words in the list. This raises further concerns that why would PTA approach a SMS Content Filtering issue without first visiting the whole issue both technically as well policy implications-wise? Again this may raise issues of direct litigation of PTA.

Should there be SMS Content Filtering?
As a personal opinion, I would discourage any form of content filtering including SMS content filtering without public consultation. PTA must put into place processes before taking decisions that affect the user/consumers of Internet and Telecom Services that make citizens aware beforehand, educate them, receive and share public comments and then consult with legislators and policy makers the possible issues pertaining to implementation well before enforcement. Even for the current issue,

Lacking Multistakeholderism and Transparency?
In the true spirit of involving the citizens of Pakistan, PTA should have first convened the public including Civil Society, Private Sector, Academia, Public Sector and both Technical and Telecom communities into a multistakeholder dialogue and then discovered whether such an action was in the best interest of the public/citizenry at large or not and more importantly in the best interest of the country both domestically and internationally.

Still actions without appropriate baseline research?
Would such an action help everyone or would it only affect a handful few. PTA so far did not produce any baseline studies on number of SMS users, number of SMS sent per year, per month or per day. There is so far no record on its website about the number of complaints received by SMS users that received abusive content through the messaging service. There were no cell phone numbers blacklisted and published online as a warning. PTA also lacks inviting public comments before publishing of any content filtering directives through either newspapers or simply its own website.

What process? This is the way forward:
PTA should have possibly carried out a rightful and meaningful process before approaching this issue. Before issuing the directive, participation of the actual service users should have been invited and most importantly from SMS users. This could also have been initiated by a SMS to all mobile users so that at least PTA could understand and have figure of the number of service users interested in such an implementation.

PTA may also have attempted to launch a request for comments through either SMS on all networks or launched an online survey posted through its website. After a thorough study of comments from the public and especially SMS users, PTA should have first publicly shared its intent and then gone forward with official implementation once the public comment would have been positive.

The world went crazy because of us! Again?
This issue has really gotten out of hands. It is a global understanding across Internet and Mobile Technology Public Policy circles that any issue pertaining to digital content filtering is a very big threat to human rights and risks violating citizens’ state-recognized civil liberties including but not limited to free speech, press freedoms, right to privacy, right to religious views, and association. The current directive has been perceived both domestically and globally an infringement upon afore mentioned civil liberties of Pakistani citizens.

Help stop the threat of Content Filtering everywhere!
It is worth mentioning here that with the plethora of circumvention and censorship technologies available both free and commercially may encourage and actually trigger a competitive content filtering industry posing a major threat to the privacy and human rights of the citizens of Pakistan. SMS content filtering is an example of limiting or preventing access to our nation’s public communications infrastructure while it may also cause violation of an individual Pakistan’s fundamental right of freedom of expression and access to information.

Privacy Concerns
As far as privacy of Pakistani citizens in terms of SMS usage is concerned, PTA should consider ensuring preservation of Privacy and the current threat around this directive is that that SMS Content Filtering Directive gives all Mobile Operators the unprecedented and uncontestable right to expose user SMS content to third party and third party applications and in this case the Mobile Operators themselves to tamper and obstruct user generated content transmitted in a private manner. Basically PTA may have failed to realize what it is unleashing from its basket of regulations.

Global example of networked ridicule!
Finally, I am of the understanding that so far none of the Telecom/Cellular/Mobile Operators have actually implemented this directive whereas PTA should also take some more time to reconsider the enforcement of its directive despite withdrawal. The oceanic wave of criticism and ridicule generated both domestically and by the global media may actually provoke reconsideration whether such a directive should be actually and fully enforced or not.

From wider discussions during various interactions with the technical and general community, it is perceived that PTA may have taken actions in the past that have actually sent out a message locally and globally raising eyebrows displaying lack of multistakeholder involvement and general public consultation while devising and implementing its telecoms related regulatory enforcement activities especially surround Internet Content Usage and now issues pertaining to SMS content sharing.

Stop scaring our own poor citizens!
It is advisable for PTA that it should work towards facilitating the citizens of Pakistan and not provoke inciting threats and fear because digital communication both on the Internet and through Mobile Technology is the backbone of all social, economic and democratic activities in Pakistan and a fundamental citizen’s right to access information today.

Sleep on it!
There is a further need to create Internet and Mobile Technology Consumer Rights Groups in Pakistan that can have open and direct dialogues with PTA without feeling threatened or pressurized by the authorities so that transparent and accountable activities can be implemented by the regulator while helping and informing PTA before it attempts to create the global chaos and unneeded attention it has recently created through this directive. The regular public monitoring of PTA is in the mutual benefit of all Pakistani citizens as well as the regulator.


Thursday, February 4, 2010

Bringing Urdu based Local Content and Knowledge Online for the Internet & Mobile/iPad platforms

I seek ideas from my readers and support for this effort so please read on and blog about it and share it with others as well contributing where ever and however you can.

The Issue at Hand:
Our actions reflect our attitudes portraying local content as a critical factor of our culture and an attitude that portrays our future  as well as the technology culture that prevails here. A key pressing issue with relevance to both the local Internet and Mobile Technology scenario in Pakistan has been availability of local content and making the local content widely accessible to the community at large across Pakistan and the entire world using a variety of currently available technology platforms.

As an initial effort to find a possible solution to the problem of local content availability in Urdu Language, we are attempting to bring some valuable information and knowledge in the form of local content to a broad Urdu Language fluent, reading, speaking, writing community online where I am teaming up with friends who are adding considerable critical value to solving this issue that prevails in the Pakistani and global Urdu online world and converged ICT environments.

Online Urdu Encyclopaedia 
(Being Multi-platform available with Converged Content)

We are embarking upon a project (in requirements research phase with content completely available in local language) to introduce online (through) the Internet and Mobile Platforms as well as Tablet Platforms (iPhone, ipad and Android) and pre-flashed (ROM) Netbooks/Low-Cost Laptops, a full length, locally written Urdu
Encyclopaedia (completed/compiled/edited by our Writer Team Member who developed it from 100 self written Urdu material-over 10 years) over an overwhelming long period of four plus years consisting of 2.5
Million Words (25 Lac), 25020 Pages, 7000 Images, 2 Volumes (typed on A4 page settings). It comprises of 17000 knowledge and information entries searchable by an index.

This Urdu Encyclopaedia will be further transcribed into Urdu Language Audio with the availability of Video based content that describes the content in its full length and true nature to accommodate users. The output of this project is multi-fold in terms of the content accessibility and delivery options we are planning to churn out as deliverables. It will create a converged environment overtime for presenting updated knowledge that is usable through reading, listening and visuals for both social and economic awareness, education, knowledge application in various fields, higher education, competitive exams, expert resources and endless Urdu language options.

We are currently working with a team of volunteers including the writer, researching the various technical/non-technical issues and options involved as well as the feasibility. We are planning to go forward with Free & Open Source Software technology platforms with an exception of iPhone/iPad Apple Software Development Kits. The content will be available globally under a Creative Commons Licence as well as indexed in the International Library Catalogues with a standardized code facility that is usually freely available. Possible content management technology platforms are Media Wiki or Drupal CMS.

This enormous project requires a great deal of initial human effort, local content conversion (specifications of the content already given) to be presentable through various platforms, various technical expertise, equipment and financial support to become a true reality and we are at the tip of this iceberg but we know it is achievable
otherwise I would not have shared it here. There will be an integrated promotional campaign for both offline and online awareness of the availability with the team members moving about most of the cities, districts and rural regions introducing this facility.

This project will not be a one time activity but will enable us to send into the online space 100 publications by the same Writer and Author plus other latest publications that require translation into Urdu. The first role out of the initial alphabetically indexed content is expected this year. The other publication that I am sponsoring is a freely downloadable/printable/compiled book on Wireless Networking in Urdu with an associated content website.

Initial Immediate Requirements - Please Contribute Now!

* We are still attempting it but would love to welcome, present and acknowledge our supporters, sponsors and contributors from this initial point onwards whether individuals, groups or organizations:


  1. Making available the Encyclopaedia through SMS search with Ufone, Telenor, China Mobile, Warid, V-PTCL, Worldcall 
  2. Apple iPad Membership Support to get the SDK and associated development instructions from http://www.apple.com/ipad/sdk/
  3. Apple iPad - Qty: 2 when it is launched next month for testing and distribution of Urdu Encyclopaedia for iPad 
  4. Amazon Kindle - Qty: 2 we can't download its SDK or PC Simulator either since you can't access those from Pakistan 
  5. Sponsored Cloud Space for a minimum of 5 years for full completion of the project (plus associated 99 projects) with an affirmation of commitment to continue the project for its content life? 
  6. Financial Sponsors/Small Supporters/Contributions/Donations from established Local Entrepreneurs and Overseas Pakistani IT/ICT Community, Local Public Sector ICT Organizations and Corporate Sector contributors such as Telecom Operators, P@SHA, PSEB, Universal Service Fund etc. 
  7. Suggestions for conversion of thousands of Urdu Inpage Software Text files to simplified text or standardized Web Fonts.
  8. Suggestions for creating an online searchable index
  9. Volunteers for constituting an Advisory Committee, Out Reach and Awareness Committee, Technical Advisory Committee, Sponsorship Committee, Local Content Expert Group etc.

Thanking you in advance for any feedback ahead of time and hope to be in contact with some of you soon to receive or share information.