再谈《Django 限制访问频率》

之前提到使用ratelimit来限制访问频率,我的目的是根据用户来限制访问频率,但是实际上通过下面的代码并没有达到效果,如果用多个浏览器进行同时刷新,会存在跳过限制的情况

@ratelimit(key='user', rate='1/8s', block=True, method=('POST'))

本来是不想重复造轮子的,但是由于这个轮子不大好用,于是只好重新造一个,基于redis可以使用下面的代码来实现(ttl为限制时长):

def set_method_limit(method_name, player_id, ttl):
    cash_name = 'RATELIMIT::METHOD=' + method_name + 'PLAYERID=' + str(player_id)
    cache.set(cash_name, method_name, ttl)
def check_is_limit(method_name, player_id):
    cash_name = 'RATELIMIT::METHOD=' + method_name + 'PLAYERID=' + str(player_id)
    if cash_name in cache:
        return True
    return False
def redis_ratelimit(method='ALL', block=False, ttl=5):
    def decorator(fn):
        @wraps(fn)
        def _wrapped(*args, **kw):
            # Work as a CBV method decorator.
            request = args[0]
            auth = request.META.get('HTTP_AUTHORIZATION', 'UNKNOWN')
            # 获取用户id 如果失败则用token
            try:
                auth = request.user.id
                print('PID= ' + str(auth))
            except:
                pass
            token = str(auth).split(' ')[-1]
            if check_is_limit(method, token) and block:
                content = {
                    'status': 403,
                    'message': '大侠喝口茶,小女子给你讲个故事如何?\r\n 从前有座山,山上有座庙……',
                }
                return JsonResponse(content)
                # raise Redis_Rate_Limit()
            set_method_limit(method, token, ttl)
            return fn(*args, **kw)

        return _wrapped

    return decorator

使用方法和retalimit一致:

@api_view(['POST', 'GET'])
@redis_ratelimit(method='api_test', block=True, ttl=10)
@csrf_exempt
def api_test(request):
    """
    测试接口
    http://192.168.1.195:8006/rest-api/battle/api-test/


    :return: 普通数据测试
    """
    return json_response_message(status=API_SUCCESS, message=timezone.now().date())

redis 安装:

pip3 install django-redis
Continue Reading

ubuntu 18.04 pip3 install mysqlclient

不知道是阿里云的问题还是ubuntu本身的问题,今天安装mysqlclient提示:

/usr/bin/ld: cannot find -lssl
/usr/bin/ld: cannot find -lcrypto
collect2: error: ld returned 1 exit status
error: command ‘x86_64-linux-gnu-gcc’ failed with exit status 1

网上搜了一下没有发现类似的错误信息,于是转换思路直接搜索: /usr/bin/ld: cannot find -lssl 在这篇文章看到了解决方案:

https://blog.51cto.com/eminzhang/1285705

Continue Reading

Django 限制访问频率

最近做了一个系统由于部分接口需要进行耗时操作,因而不希望用户进行频繁访问,需要进行访问频率限制。如果要自己实现一个访问限制功能相对来说也不会太复杂,并且网上有各种代码可以参考。如果自己不想实现这个代码可以使用 Django Ratelimit

Django Ratelimit is a ratelimiting decorator for Django views.

https://travis-ci.org/jsocol/django-ratelimit.png?branch=master
Code: https://github.com/jsocol/django-ratelimit
License: Apache Software License
Issues: https://github.com/jsocol/django-ratelimit/issues
Documentation: http://django-ratelimit.readthedocs.org/

使用方法也相对来说比较简单:

@ratelimit(key='ip', rate='5/m')
def myview(request):
    # Will be true if the same IP makes more than 5 POST
    # requests/minute.
    was_limited = getattr(request, 'limited', False)
    return HttpResponse()

@ratelimit(key='ip', rate='5/m', block=True)
def myview(request):
    # If the same IP makes >5 reqs/min, will raise Ratelimited
    return HttpResponse()

@ratelimit(key='post:username', rate='5/m', method=['GET', 'POST'])
def login(request):
    # If the same username is used >5 times/min, this will be True.
    # The `username` value will come from GET or POST, determined by the
    # request method.
    was_limited = getattr(request, 'limited', False)
    return HttpResponse()

@ratelimit(key='post:username', rate='5/m')
@ratelimit(key='post:tenant', rate='5/m')
def login(request):
    # Use multiple keys by stacking decorators.
    return HttpResponse()

@ratelimit(key='get:q', rate='5/m')
@ratelimit(key='post:q', rate='5/m')
def search(request):
    # These two decorators combine to form one rate limit: the same search
    # query can only be tried 5 times a minute, regardless of the request
    # method (GET or POST)
    return HttpResponse()

Continue Reading

Django admin Foreignkey ManyToMany list_display展示

class Ghost(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    speed = models.IntegerField(default=0, help_text='速度')
    name = models.CharField(max_length=64, help_text='名称')


class InstanceTask(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    name = models.CharField(max_length=64, help_text='副本名称')

class InstanceTaskMap(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    name = models.CharField(max_length=64, help_text='地图名称')
    ghosts = models.ManyToManyField(Ghost, help_text='Ghost')
    instance_task = models.ForeignKey(InstanceTask, related_name='instancetask_instancetaskmap', blank=True, null=True,
                                      help_text='副本任务', on_delete=models.SET_NULL)

对于上面的model,如果要在django admin中展示ghosts信息,那么在list_display中直接加入’ghosts’ 会报下面的错误:The value of ‘list_display[1]’ must not be a ManyToManyField.

如果要解决这个问题可以使用下面的代码来展示:

class InstanceTaskMapAdmin(admin.ModelAdmin):
    list_display = ('name', 'instance_task', 'id', 'index', 'get_ghost_name', 'introduction')

    # https://blog.csdn.net/weixin_42134789/article/details/83686664
    def get_ghost_name(self, obj):
        ghost_list = []
        for g in obj.ghosts.all():
            ghost_list.append(g.ghost.name)
        return ','.join(ghost_list)

    get_ghost_name.short_description = "Ghosts" 

如果需要更丰富的信息可以参考上面代码注释中的链接。

Continue Reading

Django REST framework foreignkey 序列化

Django REST framework is a powerful and flexible toolkit for building Web APIs.

Some reasons you might want to use REST framework:

之前虽然也用了Django REST framework 但是序列化函数基本都是自己写的,并没有用框架带的序列化函数。这次不想在搞的那么麻烦,于是使用Django REST framework带的序列化函数。

但是在序列化foreignkey的时候却发现只有id,其余的数据没有。

model定义:

class PlayerGoodsItem(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    goods_item = models.ForeignKey(GoodsItem, related_name='goodsitem_playergoodsitem', help_text='商品信息',
                                   on_delete=models.CASCADE)

序列化代码:

class PlayerGoodsItemSerializer(serializers.ModelSerializer):
    class Meta:
        model = PlayerGoodsItem
        fields = "__all__"

Continue Reading

由apscheduler引发的django.db.utils.InternalError: (1054, u”Unknown column ‘rms.go_datetime’ in ‘field list'”)

apscheduler.scheduler:INFO 2019-03-01 10:22:53,849 Adding job tentatively -- it will be properly scheduled when the scheduler starts
apscheduler.scheduler:INFO 2019-03-01 10:22:53,849 Adding job tentatively -- it will be properly scheduled when the scheduler starts
Do first job!
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\core\management\__init__.py", line 364, in execute_from_command_line
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\utils\functional.py", line 35, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\urls\resolvers.py", line 407, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\utils\functional.py", line 35, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\urls\resolvers.py", line 400, in urlconf_module
return import_module(self.urlconf_name)
File "C:\Python27\Lib\importlib\__init__.py", line 37, in import_module
__import__(name)
File "F:\PyCharmProjects\B\B\B\urls.py", line 433, in <module>
import td.jobs # NOQA @isort:skip
File "F:\PyCharmProjects\B\B\td\jobs.py", line 174, in <module>
deal_with_first_jobs()
File "F:\PyCharmProjects\B\B\td\jobs.py", line 88, in deal_with_first_jobs
send_wechat_mini_app_push_message(rmds)
File "F:\PyCharmProjects\B\B\td\push.py", line 201, in send_wechat_mini_app_push_message
for rmd in rmds:
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\db\models\query.py", line 250, in __iter__
self._fetch_all()
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\db\models\query.py", line 1121, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\db\models\query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
File "F:\PyCharmProjects\B\venv\lib\site-packages\django\db\models\sql\compiler.py", line 899, in execute_sql
raise original_exception
django.db.utils.InternalError: (1054, u"Unknown column 'rms.go_datetime' in 'field list'")

刚开始以为是代码写错了,后来发现并不是,出错的地方在jobs.py中所以其实并不是代码的问题,而是在jobs中为了能够在服务启动的时候发送上一次运行停止之后遗漏的任务导致的。所以如果使用了apscheduler可以尝试将jobs.py清空再次尝试。

Django APScheduler + uwsgi 定时任务重复运行

基于某些原因可能在开发的时候通过django的manage.py运行定时任务没有任何的问题,但是一旦到了线上环境通过nginx+uwsgi来运行就会发现定时任务不断的重复执行,并且基本都执行失败了。发生这个问题的原因在于uwsgi启动了多个进程来提供服务,于是每次启动的时候定时任务都会跟着再启动一次,于是有4个进程的话,对应的服务就会启动4次,除了第一次可能执行成功后面的基本都会挂掉。

要解决这个问题其实也不难,只要保证在第一次启动的时候添加定时任务并且执行,以后启动的进程不再处理定时任务即可。但是在这种条件下通过python的进程互斥其实貌似并不是非常好使,具体可以看这个:

uWSGI employs some tricks which disable the Global Interpreter Lock and with it, the use of threads which are vital to the operation of APScheduler. To fix this, you need to re-enable the GIL using the --enable-threads switch. See the uWSGI documentation for more details.

Also, assuming that you will run more than one worker process (as you typically would in production), you should also read the next section.

https://apscheduler.readthedocs.io/en/latest/faq.html#how-can-i-use-apscheduler-with-uwsgi

Continue Reading