0

I have a dataset containing names in both English and other languages and mixed too. Is their any way to remove data containing other than English language?

Eg:

enter image description here

2
  • 1
    See here how to do it with Field calculator: gis.stackexchange.com/a/450658/88814
    – Babel
    Feb 26 at 10:48
  • I assume this is really about differentiating text scripts and not about differentiating languages because differentiating Latin from Arabic or Han is one thing, but differentiating English from French or Spanish is something entirely different.
    – bixb0012
    Feb 27 at 3:49

2 Answers 2

3

You can use regular expressions. I dont know how to in Field Calculator so I use Python.

This doesnt identify language, just the characters a-Z.

See Extracting only characters from a string in Python

import re

#Change these three lines to match your layer and field names
layer = QgsProject.instance().mapLayersByName("plingplong")[0]
field_to_read = "fielda"
field_to_update = "Required N"

fieldindex = layer.fields().indexFromName(field_to_update) #Find the index of the field to update
new_attributes = {} 
pattern = r"(?i)\b[a-z]+\b"
for feature in layer.getFeatures():
    words = ' '.join(re.findall(pattern, feature[field_to_read]))
    #print(words)
    #hello
    #this is a text
    new_attributes[feature.id()]={fieldindex:words}

#new_attributes is now, each features id: {index of field to update: new text} 
#{0: {1: 'hello'}, 1: {1: 'this is a text'}}

layer.dataProvider().changeAttributeValues(new_attributes)

enter image description here

2
  • Getting this error :Traceback (most recent call last): File "C:\PROGRA~1\QGIS31~1.11\apps\Python39\lib\code.py", line 90, in runcode exec(code, self.locals) File "<input>", line 1, in <module> File "<string>", line 4, in <module> IndexError: list index out of range
    – Bruno B
    Feb 28 at 5:09
  • Have you changed "plingplong" to the name of your layer? That error usually occur when it cant find the layer by the layer name your searching for
    – BERA
    Feb 28 at 6:15
3

Use regular expressions with Field Calculator. Based on regular expressions \\p{Latin} (to match all latin characters) and \\s (to match whitespaces), use regex_replace() function to delete everything else (see here for details):

regexp_replace ("Actual Name",'([^\\p{Latin}|\\s])','')

This expression keeps latin characters and white spaces from the input and deletes everything else.

4
  • English uses Latin characters, but Latin characters doesn't necessarily mean English. That said, I am fairly sure the OP meant Latin characters and not just English (none of the examples showed English and French or some other latin language mixed), so this is definitely the closest one can get using regular expressions only.
    – bixb0012
    Feb 27 at 2:01
  • @babel formula didnt filter the listed words in the question
    – Bruno B
    Feb 28 at 5:03
  • Provide sample data
    – Babel
    Feb 28 at 6:03
  • @bixb0012 correct, this does recognize alphabets, not languages. Recognizing languages only based on placenames is virtually impossible. Paris can stand for the capital of France and than can be cosidered to be "french", but Paris is also the name of different towns in the US (like in Idaho), Canada (Ontario), Denmark etc., see en.wikipedia.org/wiki/Paris_(disambiguation) and in these cases, the placename can be considered to be english, danish etc. So really impossible to recognize "languages". How should anyone being able to recognize the language of placenames like Arel?
    – Babel
    Feb 28 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.