韩国美女模特爬虫

对于美女的热爱无法自拔 😆 ,经常会去搜索一些美女图片,下载下来,然后找时间慢慢欣赏。主要用途是用作电脑桌面手机桌面,通常会百度或者bing去搜索下找到图片下载。相对来说能够直接用作桌面的图片并不多,多数是尺寸问题,并不是十分合适。但是即使不能直接用,可以用ps修改下图片尺寸,或者欣赏也是好的啊。 🙂 

以前曾经从一个网站mzitu.com 爬了一些图片,但是最近访问的时候却发现网站挂了~~

Continue Reading

Django 限制访问频率

最近做了一个系统由于部分接口需要进行耗时操作,因而不希望用户进行频繁访问,需要进行访问频率限制。如果要自己实现一个访问限制功能相对来说也不会太复杂,并且网上有各种代码可以参考。如果自己不想实现这个代码可以使用 Django Ratelimit

Django Ratelimit is a ratelimiting decorator for Django views.

https://travis-ci.org/jsocol/django-ratelimit.png?branch=master
Code: https://github.com/jsocol/django-ratelimit
License: Apache Software License
Issues: https://github.com/jsocol/django-ratelimit/issues
Documentation: http://django-ratelimit.readthedocs.org/

使用方法也相对来说比较简单:

@ratelimit(key='ip', rate='5/m')
def myview(request):
    # Will be true if the same IP makes more than 5 POST
    # requests/minute.
    was_limited = getattr(request, 'limited', False)
    return HttpResponse()

@ratelimit(key='ip', rate='5/m', block=True)
def myview(request):
    # If the same IP makes >5 reqs/min, will raise Ratelimited
    return HttpResponse()

@ratelimit(key='post:username', rate='5/m', method=['GET', 'POST'])
def login(request):
    # If the same username is used >5 times/min, this will be True.
    # The `username` value will come from GET or POST, determined by the
    # request method.
    was_limited = getattr(request, 'limited', False)
    return HttpResponse()

@ratelimit(key='post:username', rate='5/m')
@ratelimit(key='post:tenant', rate='5/m')
def login(request):
    # Use multiple keys by stacking decorators.
    return HttpResponse()

@ratelimit(key='get:q', rate='5/m')
@ratelimit(key='post:q', rate='5/m')
def search(request):
    # These two decorators combine to form one rate limit: the same search
    # query can only be tried 5 times a minute, regardless of the request
    # method (GET or POST)
    return HttpResponse()

Continue Reading

Django admin Foreignkey ManyToMany list_display展示

class Ghost(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    speed = models.IntegerField(default=0, help_text='速度')
    name = models.CharField(max_length=64, help_text='名称')


class InstanceTask(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    name = models.CharField(max_length=64, help_text='副本名称')

class InstanceTaskMap(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    name = models.CharField(max_length=64, help_text='地图名称')
    ghosts = models.ManyToManyField(Ghost, help_text='Ghost')
    instance_task = models.ForeignKey(InstanceTask, related_name='instancetask_instancetaskmap', blank=True, null=True,
                                      help_text='副本任务', on_delete=models.SET_NULL)

对于上面的model,如果要在django admin中展示ghosts信息,那么在list_display中直接加入’ghosts’ 会报下面的错误:The value of ‘list_display[1]’ must not be a ManyToManyField.

如果要解决这个问题可以使用下面的代码来展示:

class InstanceTaskMapAdmin(admin.ModelAdmin):
    list_display = ('name', 'instance_task', 'id', 'index', 'get_ghost_name', 'introduction')

    # https://blog.csdn.net/weixin_42134789/article/details/83686664
    def get_ghost_name(self, obj):
        ghost_list = []
        for g in obj.ghosts.all():
            ghost_list.append(g.ghost.name)
        return ','.join(ghost_list)

    get_ghost_name.short_description = "Ghosts" 

如果需要更丰富的信息可以参考上面代码注释中的链接。

Continue Reading

Django REST framework foreignkey 序列化

Django REST framework is a powerful and flexible toolkit for building Web APIs.

Some reasons you might want to use REST framework:

之前虽然也用了Django REST framework 但是序列化函数基本都是自己写的,并没有用框架带的序列化函数。这次不想在搞的那么麻烦,于是使用Django REST framework带的序列化函数。

但是在序列化foreignkey的时候却发现只有id,其余的数据没有。

model定义:

class PlayerGoodsItem(models.Model):
    create = models.DateTimeField(default=timezone.now, help_text='创建时间')
    update = models.DateTimeField(auto_now=True, help_text='修改时间')
    goods_item = models.ForeignKey(GoodsItem, related_name='goodsitem_playergoodsitem', help_text='商品信息',
                                   on_delete=models.CASCADE)

序列化代码:

class PlayerGoodsItemSerializer(serializers.ModelSerializer):
    class Meta:
        model = PlayerGoodsItem
        fields = "__all__"

Continue Reading

基于DFA的敏感词过滤

在计算理论中,确定有限状态自动机或确定有限自动机(英语:deterministic finite automaton, DFA)是一个能实现状态转移的自动机。对于一个给定的属于该自动机的状态和一个属于该自动机字母表{\displaystyle \Sigma }Σ的字符,它都能根据事先给定的转移函数转移到下一个状态

DFA算法

DFA((Deterministic Finite automation))确定性的有穷状态自动机: 从一个状态输入一个字符集合能到达下一个确定的状态。如图:

 
dfa_1.png

如上图当AB状态输入a得到状态aB,状态aB输入b得到状态ab; 状态AB输入b得到状态Ab,状态Ab输入a得到状态ab。

Continue Reading

jupyter notebook 调整字体 以及matplotlib显示中文

原生的jupyter theme看起来比较蛋疼,尤其是字体和字号。为了修改这个配置可以安装 jupyter theme。

项目链接: https://github.com/dunovank/jupyter-themes 如果不喜欢英文可以参考这个链接:https://www.jianshu.com/p/6de5f6cce06d

上面的样式对应的配置命令:
jt  -f fira -fs 11 -cellw 90% -ofs 11 -dfs 11 -T -t solarizedl

除此之外matplotlib 默认不支持中文显示,主要是字体问题,可以通过下面的代码配置来让matplotlib 支持中文

from matplotlib import pyplot as plt
%matplotlib inline
font = {'family' : 'MicroSoft YaHei',
'weight' : 'bold',
'size' : 10}
plt.rc("font", **font)

实际效果,另外还可以使用altair ,altair 默认支持中文显示 https://altair-viz.github.io

基于RandomForestClassifier的titanic生存概率分析

The Challenge

The sinking of the Titanic is one of the most infamous shipwrecks in history.

On April 15, 1912, during her maiden voyage, the widely considered “unsinkable” RMS Titanic sank after colliding with an iceberg. Unfortunately, there weren’t enough lifeboats for everyone onboard, resulting in the death of 1502 out of 2224 passengers and crew.

While there was some element of luck involved in surviving, it seems some groups of people were more likely to survive than others.

In this challenge, we ask you to build a predictive model that answers the question: “what sorts of people were more likely to survive?” using passenger data (ie name, age, gender, socio-economic class, etc).

这个是kaggle上的一个基础项目,目的是探测泰坦尼克号上的人员的生存概率,项目地址:https://www.kaggle.com/c/titanic

网上基于这个项目其实可以找到各种各样的解决方案,我也尝试了不同的做法。但是实际的效果并不是十分好,个人尝试最好的成绩是0.78468,一次是基于深度神经网络,另外一次就是基于当前的随机森林的模型。

另外还可以看到一系列score为1的提交,这些不知道是怎么做到的,真是太tm牛了~~

Continue Reading

CUDNN_STATUS_NOT_INITIALIZED

自从装好tensorflow-gpu 之后其实一直没怎么用,今天跑代码的时候才发现安装的有问题:

测试代码如下:

from sklearn.datasets import load_sample_image
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf

if __name__ == '__main__':
    # Load sample images
    china = load_sample_image("china.jpg")
    flower = load_sample_image("flower.jpg")
    dataset = np.array([china, flower], dtype=np.float32)
    batch_size, height, width, channels = dataset.shape
    # Create 2 filters
    filters = np.zeros(shape=(7, 7, channels, 2), dtype=np.float32)
    filters[:, 3, :, 0] = 1 # vertical line
    filters[3, :, :, 1] = 1 # horizontal line
    # Create a graph with input X plus a convolutional layer applying the 2 filters
    X = tf.placeholder(tf.float32, shape=(None, height, width, channels))
    convolution = tf.nn.conv2d(X, filters, strides=[1,2,2,1], padding="SAME")
    with tf.Session() as sess:
        output = sess.run(convolution, feed_dict={X: dataset})
    plt.imshow(output[0, :, :, 1], cmap="gray") # plot 1st image's 2nd feature map
    plt.show()
Continue Reading